Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Research Metrics


Published on

Faculty Development Workshop

  • Be the first to comment

Research Metrics

  1. 1. Research Metrics <br />Nazi Torabi<br />Research & Instructional Librarian <br />Allyn & Betty Taylor Library<br />April 27, 2011<br /><br />1<br />
  2. 2. Objectives <br />To provide an overview of various bibliometrics products<br />Have a good understanding of limitation of each product <br />How to calculate your own h-index<br />2<br />
  3. 3. 3<br />
  4. 4. Why Research Metrics?<br />Get published in a good journal<br />Application for tenure, promotion, or grant funding<br />Measure the impact of an individual’s research<br />4<br />
  5. 5. The Metrics<br />Publication count measures quantity but not quality (impact) <br />Citation counts <br /><ul><li>Total number of citation count
  6. 6. Total number of citation count in particular time span
  7. 7. Total number of citation per paper
  8. 8. Number of papers cited more than n times
  9. 9. Number of citations to the n most cited papers</li></ul>Normalized citation counts <br />5<br />
  10. 10. Journal Citation Report<br /><ul><li>Key metric: Journal Impact Factor (IF)
  11. 11. Journal ranking by discipline
  12. 12. Citing pattern analysis in discipline </li></ul>6<br />
  13. 13. Impact Factor<br />Number of times articles published in 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2008 and 2009<br />2010 Journal IF = <br />i.e. An Impact Factor of 1.0 in 2010 means that, on average, the articles published in 2008 and 2009 have been cited one time. Self-citation is included.<br />7<br />
  14. 14. Journal Immediacy Index<br />Number of times articles published in 2010 were cited by indexed journals during 2010<br />Total number of "citable items" published by that journal in 2010<br />2010 Journal Immediacy =<br />Index<br /> i.e. The journal Immediacy Index indicates how quickly articles in a journal are cited.<br />JCR <br />8<br />
  15. 15. For 2009, the journal CHEMBIOCHEM has an Impact Factor of 3.824.<br />9<br />
  16. 16. Journal Ranking<br />The Journal Ranking table shows the ranking of the current journal in its subject categories based on the journal IF. <br />Data for Total Journals in Category and Quartile in Category are not available for JCR years before 2003. <br />10<br />
  17. 17. IF & its shortcomings <br />Citation pattern is discipline specific <br />Time span<br />Citation frequency<br />differences in citation norms<br />Sleeping beauties<br />Self-Citation and other!!!<br />Negative citations <br />Title and journal format changes<br />A questionable validity<br />11<br />
  18. 18. Impact Factor<br />Number of times articles published in 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2008 and 2009<br />2010 Journal IF = <br />i.e. An Impact Factor of 1.0 in 2010 means that, on average, the articles published in 2008 and 2009 have been cited one time. Self-citation is included.<br />12<br />
  19. 19. Probability density function for journals in subject categories of ENERGY & FUELS – data from JCR<br />-5 -4 -3 -2 -1 0 1 2 3 4 5<br />Normal distribution<br />Pareto distribution<br />13<br />
  20. 20.<br />14<br />
  21. 21. Citation Networks & Network Effects <br />15<br />
  22. 22. Scholarship is the flow of idea. <br />16<br />
  23. 23. High School Dating Network<br />Bearman, Peter S., James Moody, and Katherine Stovel. "Chains of Affection: The Structure of Adolescent Romantic and Sexual Networks." American Journal of Sociology 110, no. 1 (July 2004): 44-91. <br />17<br />
  24. 24. Which network you would rather be part of?<br />West, J (2010). The EigenfactorTM Metrics: a network approach to assessing scholarly journals.<br />18<br />A<br />B<br />
  25. 25. The Rich gets Richer <br /><ul><li>Journals are ranked according to the citation network.
  26. 26. Journals are influential when they are cited by other influential journals.
  27. 27. “The Eigenfactor™ score of a journal is an estimate of the percentage of time that library users spend with that journal.”</li></ul>19<br />
  28. 28.<br />20<br />
  29. 29. Article Influence™ Score<br />Journal’s Eigenfactor Score (5yrs)<br />Number of articles in the journal<br />Article Influence™ = <br />Score<br /><br />21<br />
  30. 30. How can we better evaluate the scholarly literature ?<br />How can we better navigate the scholarly literature?<br />22<br />
  31. 31. CWTS Journal Indicators<br /><ul><li>Key metric: Source Normalized Impact per Paper (SNIP)
  32. 32. Journal ranking by discipline
  33. 33. Author analysis
  34. 34. Article analysis</li></ul>23<br />
  35. 35. Raw Impact per Paper (RIP)<br />Number of times articles published in 2007, 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2007, 2008 and 2009<br />2010 Journal RIP= <br />24<br />
  36. 36. Source normalized impact per paper (SNIP) <br /> A journal’s Source Normalized Impact per Paper (SNIP) <br /> =<br />Raw Impact per Paper published in the journal (RIP) <br /> ÷<br />Relative Database Citation Potential (RDCP) in the journal’s subfield<br />i.e. It corrects for differences in the frequency of citation across research fields.<br /><br />25<br />CWTS<br />
  37. 37. SCImago<br /><ul><li>Key metric: ScImago Journal Ranking (SJR)
  38. 38. Journal and Country ranking by discipline</li></ul>26<br />Scopus<br />
  39. 39. Journal Ranking Summary <br />Adapted and modified from: González-Pereir , B. et al. “The SJR indicator: A new indicator of journals' scientific prestige”.<br />27<br />
  40. 40. Exercise : Journal Ranking<br />What is the Impact Factor Trend in the last 5 yearsfor the Am J Bioeth? Tip: you need to find the journal’s full title first.<br />You are thinking of submitting a manuscript to the journal of MOLECULAR MEDICINE. How do you rank this journal in its appropriate subject category? Is it better to present this journal using IF or EigenFactor? <br />Using Scopus, compare the journal of MOLECULAR MEDICINE with the journal of CURRENT MOLECULAR MEDICINE. Which journal might be a better candidate for your article?<br />28<br />
  41. 41. h-index<br /><ul><li>h measures the cumulative impact of a researcher's output by looking at the amount of citation his/her work has received.
  42. 42. i.e. If a researcher has an h-index of 25 if 25 of their papers have been cited at least 25 times (See the graph in the handout).</li></ul>29<br />
  43. 43. h-index shortcomings <br />Early career researchers <br />Sleeping beauties<br />The balance between number of publications vs. number of citations each receives<br />Databases are not error free<br />Inconsistency between different metric tools (See the handout). <br />30<br />5 articles published – each cited 5 times – h-index = 5<br />2 articles published – each cited 20 times – h-index = 2 <br />
  44. 44. h-index Variants<br />g-index: more weight is given to the highest cited papers <br />The individual h-index: accounts for co-authorship in calculating impact by giving less weight to such papers<br />The contemporary h-index: less weight to older cited papers <br />The age-weighted citation rate: accounts for the age of papers <br />Tarma Software Research (2010). Publish or Perish User’s Manual<br />31<br />
  45. 45. ArticlesRanking <br />F1000<br />Scopus<br />POP<br />WOS<br />32<br />
  46. 46. Web OF Science<br />33<br />
  47. 47. Citation Report from WOS<br /> The Citation Report provides aggregate citation statistics for a set of search results. These statistics include:<br /><ul><li>The total number of times all items have been cited
  48. 48. The average number of times an item has been cited
  49. 49. The number of times an item has been cited each year
  50. 50. The average number of times an item has been cited in a year
  51. 51. More info</li></ul>34<br />
  52. 52. Scopus<br />35<br />
  53. 53. Citation overview from Scopus<br />The Citation Overview Results page lists documents that cited the document you selected from the Citation Overview.<br />You can include up to 5000 documents in a Citation Tracker. If you select more than 5000 documents only the first 5000 are included.<br />36<br />
  54. 54. Exercise - Article analysis<br />What is the most cited article written by Dr. Terry Delovitch? Tip: use both WOS and Scopus <br />37<br />
  55. 55. 38<br />
  56. 56. 39<br />
  57. 57. Publish or Perish <br />40<br />
  58. 58. Harzing's Publish or Perish <br /> Publish or Perish is a software program that retrieves and analyzes academic citations. It uses Google Scholar to obtain the raw citations, then analyzes these and presents the following statistics:<br /><ul><li>Total number of papers
  59. 59. Total number of citations
  60. 60. Average number of citations per paper
  61. 61. Average number of citations per author
  62. 62. Average number of papers per author
  63. 63. Average number of citations per year
  64. 64. h-index and related parameters
  65. 65. An analysis of the number of authors per paper.</li></ul>41<br />
  66. 66. Google Scholar vs. WOS dataset <br />Higher transparency <br />Coverage<br /><ul><li>Better coverage for unique citations
  67. 67. Not limited to journal articles published in ISI listed journals
  68. 68. Greater geographical coverage for non-US/UK based journals  
  69. 69. Better coverage of non-English sources </li></ul>Includes more comprehensive data for junior academics<br />All authors are listed<br />42<br />
  70. 70. Google Scholar vs. WOS dataset cont’<br />Accounts for minor citation errors <br />Does not account for duplications <br />Includes non-scholarly citations such as student handbooks, library guides or editorial notes.<br />Unclear coverage<br /><ul><li>uneven coverage across different fields of study
  71. 71. older materials are not well covered</li></ul>43<br />
  72. 72. Exercise - Article analysis<br />Find Bibliometricsinformation (# articles published, h-index, etc.) for Anne-WilHarzingusing the following: <br />WOS general search function: Author Finder <br />Scopus <br />Compare the results <br />44<br />
  73. 73. Download and install Publish or Perish at<br />45<br />
  74. 74. F1000<br /><ul><li>Key metric: F1000 Article Factors (FFa)
  75. 75. Medicine and Biology specific </li></ul>46<br />
  76. 76. F1000 factors <br />The F1000 Article Factor (FFa) is calculated from the highest rating awarded by a Faculty Member (FM) plus an increment for each additional rating from other FMs.<br />For example, a single article that has been evaluated by three Faculty Members, who rated it ‘Recommended’, ‘Must Read’ and ‘Must Read’, will have its FFa calculated thus:<br />8 (highest rating ‘Must Read’) + 2 (increment for ‘Must Read’) + 1 (increment for ‘Recommended’) = 11.<br />47<br />
  77. 77. Which metric tool?<br />Adapted form<br />48<br />
  78. 78. Questions<br />Need library related assistance?Please don’t hesitate to contact us:<br />Nazi Torabi<br />519-661-2111 x88992 <br /><br />IM a librarian<br /><br />49<br />