Measuring research impact with bibliometrics
•Download as PPTX, PDF•
4 likes•2,782 views
Updated 30/01/2015 This session included discussions around the value of bibliometrics for individual performance management/promotion and the REF. What are bibliometrics? Journal metrics Personal metrics Article level metrics and altmetrics
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (11)
Similar to Measuring research impact with bibliometrics
Similar to Measuring research impact with bibliometrics (20)
More from Lancaster University Library
More from Lancaster University Library (20)
Recently uploaded
Recently uploaded (20)
Measuring research impact with bibliometrics
- 1. Measuring research impact with bibliometrics Friday 30th January Tanya Williamson, Assistant Librarian
- 2. Overview Learn about the most common bibliometrics and the tools used to measure research impact and influence. 1. What are bibliometrics? 2. Journal metrics 3. Personal metrics 4. Article level metrics and altmetrics Optional practical: Have a go at finding a journal's impact factor, and calculating a researcher’s h-index. • Practical – Find journal metrics using journal citation reports • Practical – Find a researcher’s h-index using Google Scholar or Web of Science
- 3. • Bibliometrics are quantitative methods of analysing research publications and citations • Based on the assumption that citations = impact What are bibliometrics? Citation counts in Web of Science Visualisation of citation networks from InCites Journal Citation Reports Graphic from Eigenfactor.org
- 4. An attempt to compare the influence and impact of journal titles in a particular discipline. Can be used to inform where to publish, which could in turn improve research impact. Commonly used metrics available through Journal Citation Reports, part of Web of Science • Journal Impact Factor • 5 year Journal Impact Factor • Immediacy index • Eigenfactor score and • SCImago Journal rank Journal metrics
- 5. Journal Citation Reports category rank An example: Journal of Allergy and Clinical Immunology ALLERGY category rank
- 6. Journal Impact Factor JIF 2011 2012 2013 2014 Citations to 2011 publications Citations to 2012 publications The JCR year Items published Items published 2015 175 125+ 250 350+ = 2 The Journal Impact Factor is the average number of times articles from the journal published in the past two years have been cited in the JCR year. The Journal of Important Discoveries Other Journals in WoS
- 7. The Journal Impact Factor is the average number of times articles from the journal published in the past two years have been cited in the JCR year. Also available: 5 year Journal Impact Factor Journal Impact Factor An example: Journal of Allergy and Clinical Immunology
- 8. • The Immediacy Index is the average number of times an article is cited in the year it is published. • Good for comparing cutting edge research field Immediacy Index An example: Journal of Allergy and Clinical Immunology
- 9. • Based on source data from Web Of Science • Not influenced by journal self- citation Eigenfactor and Article Influence An example: Journal of Allergy and Clinical Immunology Demo of Journal Citation Reports
- 10. SCImago journal rank An example: Journal of Allergy and Clinical Immunology • SCImago Journal Rank uses source data from Scopus • Based on the Google PageRank algorithm • Weights citations from high prestige journals and attempts to balance the influence of the size of a journal Immunology and Allergy category rank
- 11. • Don’t work well for every discipline due to different publishing patterns and coverage of different publication types • Mustn’t be used to compare different disciplines Limitations of journal metrics Top scorer in Premier league is Luis Suarez with 31 goals Top scorer in NBA is Kevin Durant with 849 goals Can we conclude that Durant is a better sportsman?
- 12. • Focusing only on Journal Impact Factors could lead to you overlooking smaller, specialist publications • Dependent on the coverage in the Web of Science database • Can we infer individual impact from journal impact? Limitations of journal metrics
- 13. This video: Bibliometrics for the Individual, Prof Dermot Diamond Measure your Research Impact, Bibliometrics toolkit on YouTube Bibliometrics for individuals
- 14. Key metrics • Citation counts - uses a particular dataset to count how many citations an author has received • H-index, devised by: J. E. Hirsch in 2005 • There are several ‘improvements’ on the h-index, e.g. the g- index, the e-index, the w-index, the hfa index… Personal metrics Easy explanation: If a scholar has 20 articles which have each been cited 20 times, s/he has an h-index of 20
- 15. • Web of Science Personal metrics Demo of Web of Science
- 16. Google Scholar • A way to find your own h-index • The author needs to have a Google Scholar profile Personal metrics Publish or Perish • Uses Google Scholar data • Author and journal metrics • Accompanying book and website
- 17. • Name ambiguity – getting a comprehensive list is not easy, even in Web of Science • No one metric can capture all citations or ‘impacts’ • Early career researchers will be at a disadvantage • Citations ≠ endorsements of quality! Limitations of personal metrics Resistance against the h-index, ImpactStory
- 18. • Citation analysis in Web of Science, Google Scholar • Publishers’ interfaces often include citing articles Article level metrics Publisher’s site Google Scholar Web of Science
- 19. “Article-Level Metrics (ALMs) leverage the acceleration of research communication made possible by the networked landscape of researcher tools and services. Also by incorporating the manifold ways in which research is disseminated, these article impact indicators are made available rapidly after publication and are continually updated.” PLOS One Article Level Metrics Information
- 20. Alternative metrics or altmetrics ImpactStory: like a living publications CV. Adding metrics all the time https://impactstory.org/metrics Altmetric.com: watch social media sites, newspapers, government policy documents and other sources for mentions of scholarly articles to compile article level metrics. • Shows the broader, societal impact • More than just articles
- 21. Remember that highly cited article from before? Limitations of article level and altmetrics Source of data Citation to an article Web of Science 2021 Google Scholar 2937 Publisher’s website 1478
- 22. • Publish or Perish http://www.harzing.com/pop.htm • Measure Your Research Impact toolkit http://www.ndlr.ie/myri/ • Conference Rankings for Computer Science: http://www.core.edu.au/index.php/conference-rankings • The Ultimate Guide to Altmetrics (Researchers edition) http://impactstory.demo.libguides.com/c.php?g=211074 The University is currently considering other tools, such as Scopus, SciVal and Altmetric for Institutions. Thanks to Scott Taylor from University of Manchester for permitting me to borrow the sporting analogy from his presentation with Stephen Pearson at the Northern Collaboration Conference 2014. http://www.slideshare.net/northerncollaboration/northern- collaboration-conference-2014-40398613 Follow up resources
- 23. 1. Navigate to Journal Citation Reports from the Library Databases A-Z http://www.lancaster.ac.uk/library 2. Choose a subject category to compare journals against different metrics Practical
- 24. Find a scholar’s H-index in Google Scholar 1. Go to Google Scholar http://scholar.google.co.uk/ 2. Search for a topic of your interest 3. Choose a scholar who has a Google Scholar profile … Outside of the session, you can look into Publish or Perish software available from http://www.harzing.com/pop.htm which ingests data from Google Scholar and allows you to analyse on a variety of journal and personal metrics. Practical
Editor's Notes
- Learn about the most common bibliometrics used to measure research impact and influence. Have a go at finding a journal's impact factor and calculating the h-index.
- This is a complex area, I do not aim to teach you how to be bibliometricians – I’m not one myself! Ask: I’d like to gauge where your priorities lie for the session today. Is there something specific you hope to get out of the session?
- This is a complex area, I do not aim to teach you how to be bibliometricians – I’m not one myself! There are metrics that you can access, with source data, which can help you to gain a picture of research impact. Ideally bibliometrics should be used alongside other ‘measures’ of impact or quality, which include: -- successful funding applications -- influence on policy/society -- peer review -- analysis of public engagement We will look into a range of metrics and their limitations
- Two versions of JCR – basic and InCites (graphical) not much difference, basic is probably easier, though InCites allows you to use BOTH Sci Index and SSci Index journal impact factors in the Journal Citation Reports database which ranks journals using citation data from ISI Citation Indexes in the Web of Knowledge based on the previous 2 years 5 year impact factor which extends the citation data over 5 years h index which is a means of calculating the impact of individuals, institutions or research groups using citation data from Web of Science or Google Scholar (the Library does not currently subscribe to the Scopus database) Eigenfactor score which aims to measure total influence by the use of an algorithm which includes the journal impact factor but excludes journal self-citations SCImago
- The Impact Factor is calculated by dividing the number of citations in the JCR year by the total number of articles published in the two previous years. An Impact Factor of 1.0 means that, on average, the articles published one or two year ago have been cited one time. An Impact Factor of 2.5 means that, on average, the articles published one or two year ago have been cited two and a half times. The citing works may be articles published in the same journal. However, most citing works are from different journals, proceedings, or books indexed by Web of Science.
- Because it is a per-article average, the Immediacy Index tends to discount the advantage of large journals over small ones. However, frequently issued journals may have an advantage because an article published early in the year has a better chance of being cited than one published later in the year. Many publications that publish infrequently or late in the year have low Immediacy Indexes. For comparing journals specializing in cutting-edge research, the immediacy index can provide a useful perspective. Publications which publish infrequently or late in the year will be at a disadvantage
- Eigenfactor scores are scaled so that the sum of the Eigenfactor scores of all journals listed in Thomson's Journal Citation Reports (JCR) is 100. In 2006, the journal Nature has the highest Eigenfactor score, with a score of 1.992. The top thousand journals, as ranked by Eigenfactor score, all have Eigenfactor scores above 0.01. A journal's Article Influence score is a measure of the average influence of each of its articles over the first five years after publication. Article Influence score measures the average influence, per article, of the papers in a journal. As such, it is comparable to Thomson Scientific's widely-used Impact Factor. Article Influence scores are normalized so that the mean article in the entire Thomson Journal Citation Reports (JCR) database has an article influence of 1.00. In 2006, the top journal by Article Influence score is Annual Reviews of Immunology, with an article influence of 27.454. This means that the average article in that journal has twenty seven times the influence of the mean journal in the JCR.
- Si-MAhG-o Global data – country ranks
- Sporting analogy Is Durant 27 times better than Suarez?
- Common one is the H-index is a peculiar measure. Anybody heard of it? What does it mean? What value is placed on it? Not an average, (though you can get an average citations per article from WoS)
- Demo WoS Topic: wheezing in children Sort: most highly cited Choose author: Martinez Create Citation Report Mention that you would actually have to work a little harder to ensure that you have everything in the database by this author.
- Possible to create a Scholar account which enables you to (and others if the account is public) to view citations to your work, and citation analysis. The i10-index indicates the number of academic publications an author has written that have at least ten citations from others. It was introduced in July 2011 by Google as part of their work on Google Scholar, a search engine dedicated to academic and related papers.
- Growing area of development in bibliometrics, that goes hand in hand with the Open Access movement. Rather than using the journal metrics as a proxy for article quality, it’s now possible to split that link and focus on the actual article – Have been able to do this in Web of Science for a long time. Highlight differences in the count.
- From OA journal PLOS One (Public Library of Science). No citations from other
- Altmetric.com focuses, like traditional metrics on journal articles. ImpactStory looks at datasets, slides, videos, patents etc etc, not just articles
- (>2000 cites) Older articles won’t necessarily get fresh attention. Which source do you trust?