Research Metrics

3,600 views

Published on

Faculty Development Workshop

0 Comments
10 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,600
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
1
Comments
0
Likes
10
Embeds 0
No embeds

No notes for slide
  • A = the number of times articles published in 2008 and 2009 were cited by indexed journals during 2010B = the total number of "citable items" published by that journal in 2008 and 2009. 2010 impact factor = A/B("Citable items" are usually articles, reviews, proceedings, or notes; not editorials or Letters-to-the-Editor.)
  • http://admin-apps.isiknowledge.com/JCR/help/h_boxplot.html#category_boxplot
  • the cited half-life: the median age of the articles that were cited in Journal Citation Reports each year. For example, if a journal's half-life in 2005 is 5, that means the citations from 2001-2005 are half of all the citations from that journal in 2005, and the other half of the citations precede 2001
  • A = the number of times articles published in 2008 and 2009 were cited by indexed journals during 2010B = the total number of "citable items" published by that journal in 2008 and 2009. 2010 impact factor = A/B("Citable items" are usually articles, reviews, proceedings, or notes; not editorials or Letters-to-the-Editor.)
  • The impact factor refers to the average number of citations per paper, but this is not a normal distribution. It is rather a Bradford distribution, as predicted by theory. Being an arithmetic mean, the impact factor therefore is not a valid representation of this distribution and unfit for citation evaluation.[7]
  • It was created by Physicist Mark Newman from data published in 2004 by Peter Bearman, James Moody, and Katherine Stovel. http://www.jstor.org/stable/pdfplus/10.1086/386272.pdf?acceptTC=truehttp://mediazone.brighttalk.com/comm/ReedElsevier/ad0fb0ef1d-21688-4348-23066
  • Head developer ofeigenfactor
  • SJR = SCImago Journal Rank reflects prestige of source: value of weighted citations per documentSNIP = Source normalized impact per paper: corrects for differences in the frequency of citation across research fields4 yrs of data is needed
  • Use pubmed journal database = AMERICAN JOURNAL OF BIOETHICS = 2.5- 3.4- 3.9-4.3-4.0IF mol med = 5.02 no ImmediacyIndexIF current mol med = 5.092
  • An index to quantify an individual's scientific research output
  • Research Metrics

    1. 1. Research Metrics <br />Nazi Torabi<br />Research & Instructional Librarian <br />Allyn & Betty Taylor Library<br />April 27, 2011<br />http://www.slideshare.net/ntorabi/<br />1<br />
    2. 2. Objectives <br />To provide an overview of various bibliometrics products<br />Have a good understanding of limitation of each product <br />How to calculate your own h-index<br />2<br />
    3. 3. 3<br />
    4. 4. Why Research Metrics?<br />Get published in a good journal<br />Application for tenure, promotion, or grant funding<br />Measure the impact of an individual’s research<br />4<br />
    5. 5. The Metrics<br />Publication count measures quantity but not quality (impact) <br />Citation counts <br /><ul><li>Total number of citation count
    6. 6. Total number of citation count in particular time span
    7. 7. Total number of citation per paper
    8. 8. Number of papers cited more than n times
    9. 9. Number of citations to the n most cited papers</li></ul>Normalized citation counts <br />5<br />
    10. 10. Journal Citation Report<br /><ul><li>Key metric: Journal Impact Factor (IF)
    11. 11. Journal ranking by discipline
    12. 12. Citing pattern analysis in discipline </li></ul>6<br />
    13. 13. Impact Factor<br />Number of times articles published in 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2008 and 2009<br />2010 Journal IF = <br />i.e. An Impact Factor of 1.0 in 2010 means that, on average, the articles published in 2008 and 2009 have been cited one time. Self-citation is included.<br />7<br />
    14. 14. Journal Immediacy Index<br />Number of times articles published in 2010 were cited by indexed journals during 2010<br />Total number of "citable items" published by that journal in 2010<br />2010 Journal Immediacy =<br />Index<br /> i.e. The journal Immediacy Index indicates how quickly articles in a journal are cited.<br />JCR <br />8<br />
    15. 15. For 2009, the journal CHEMBIOCHEM has an Impact Factor of 3.824.<br />9<br />
    16. 16. Journal Ranking<br />The Journal Ranking table shows the ranking of the current journal in its subject categories based on the journal IF. <br />Data for Total Journals in Category and Quartile in Category are not available for JCR years before 2003. <br />10<br />
    17. 17. IF & its shortcomings <br />Citation pattern is discipline specific <br />Time span<br />Citation frequency<br />differences in citation norms<br />Sleeping beauties<br />Self-Citation and other!!!<br />Negative citations <br />Title and journal format changes<br />A questionable validity<br />11<br />
    18. 18. Impact Factor<br />Number of times articles published in 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2008 and 2009<br />2010 Journal IF = <br />i.e. An Impact Factor of 1.0 in 2010 means that, on average, the articles published in 2008 and 2009 have been cited one time. Self-citation is included.<br />12<br />
    19. 19. Probability density function for journals in subject categories of ENERGY & FUELS – data from JCR<br />-5 -4 -3 -2 -1 0 1 2 3 4 5<br />Normal distribution<br />Pareto distribution<br />13<br />
    20. 20. Eigenfactor.org<br />14<br />
    21. 21. Citation Networks & Network Effects <br />15<br />
    22. 22. Scholarship is the flow of idea. <br />16<br />
    23. 23. High School Dating Network<br />Bearman, Peter S., James Moody, and Katherine Stovel. "Chains of Affection: The Structure of Adolescent Romantic and Sexual Networks." American Journal of Sociology 110, no. 1 (July 2004): 44-91. <br />17<br />
    24. 24. Which network you would rather be part of?<br />West, J (2010). The EigenfactorTM Metrics: a network approach to assessing scholarly journals. http://mediazone.brighttalk.com/comm/ReedElsevier/ad0fb0ef1d-21688-4348-23066<br />18<br />A<br />B<br />
    25. 25. The Rich gets Richer <br /><ul><li>Journals are ranked according to the citation network.
    26. 26. Journals are influential when they are cited by other influential journals.
    27. 27. “The Eigenfactor™ score of a journal is an estimate of the percentage of time that library users spend with that journal.”</li></ul>19<br />
    28. 28. http://well-formed.eigenfactor.org/radial.html<br />20<br />
    29. 29. Article Influence™ Score<br />Journal’s Eigenfactor Score (5yrs)<br />Number of articles in the journal<br />Article Influence™ = <br />Score<br />http://www.eigenfactor.org/<br />21<br />
    30. 30. How can we better evaluate the scholarly literature ?<br />How can we better navigate the scholarly literature?<br />22<br />
    31. 31. CWTS Journal Indicators<br /><ul><li>Key metric: Source Normalized Impact per Paper (SNIP)
    32. 32. Journal ranking by discipline
    33. 33. Author analysis
    34. 34. Article analysis</li></ul>23<br />
    35. 35. Raw Impact per Paper (RIP)<br />Number of times articles published in 2007, 2008 and 2009 were cited by indexed journals during 2010 <br />Total number of "citable items" published by that journal in 2007, 2008 and 2009<br />2010 Journal RIP= <br />24<br />
    36. 36. Source normalized impact per paper (SNIP) <br /> A journal’s Source Normalized Impact per Paper (SNIP) <br /> =<br />Raw Impact per Paper published in the journal (RIP) <br /> ÷<br />Relative Database Citation Potential (RDCP) in the journal’s subfield<br />i.e. It corrects for differences in the frequency of citation across research fields.<br />http://www.journalindicators.com/Indicators.pdf<br />25<br />CWTS<br />
    37. 37. SCImago<br /><ul><li>Key metric: ScImago Journal Ranking (SJR)
    38. 38. Journal and Country ranking by discipline</li></ul>26<br />Scopus<br />
    39. 39. Journal Ranking Summary <br />Adapted and modified from: González-Pereir , B. et al. “The SJR indicator: A new indicator of journals' scientific prestige”. http://arxiv.org/ftp/arxiv/papers/0912/0912.4141.pdf<br />27<br />
    40. 40. Exercise : Journal Ranking<br />What is the Impact Factor Trend in the last 5 yearsfor the Am J Bioeth? Tip: you need to find the journal’s full title first.<br />You are thinking of submitting a manuscript to the journal of MOLECULAR MEDICINE. How do you rank this journal in its appropriate subject category? Is it better to present this journal using IF or EigenFactor? <br />Using Scopus, compare the journal of MOLECULAR MEDICINE with the journal of CURRENT MOLECULAR MEDICINE. Which journal might be a better candidate for your article?<br />28<br />
    41. 41. h-index<br /><ul><li>h measures the cumulative impact of a researcher's output by looking at the amount of citation his/her work has received.
    42. 42. i.e. If a researcher has an h-index of 25 if 25 of their papers have been cited at least 25 times (See the graph in the handout).</li></ul>29<br />
    43. 43. h-index shortcomings <br />Early career researchers <br />Sleeping beauties<br />The balance between number of publications vs. number of citations each receives<br />Databases are not error free<br />Inconsistency between different metric tools (See the handout). <br />30<br />5 articles published – each cited 5 times – h-index = 5<br />2 articles published – each cited 20 times – h-index = 2 <br />
    44. 44. h-index Variants<br />g-index: more weight is given to the highest cited papers <br />The individual h-index: accounts for co-authorship in calculating impact by giving less weight to such papers<br />The contemporary h-index: less weight to older cited papers <br />The age-weighted citation rate: accounts for the age of papers <br />Tarma Software Research (2010). Publish or Perish User’s Manual http://www.harzing.com/pophelp/metrics.htm#gindex<br />31<br />
    45. 45. ArticlesRanking <br />F1000<br />Scopus<br />POP<br />WOS<br />32<br />
    46. 46. Web OF Science<br />33<br />
    47. 47. Citation Report from WOS<br /> The Citation Report provides aggregate citation statistics for a set of search results. These statistics include:<br /><ul><li>The total number of times all items have been cited
    48. 48. The average number of times an item has been cited
    49. 49. The number of times an item has been cited each year
    50. 50. The average number of times an item has been cited in a year
    51. 51. More info</li></ul>34<br />
    52. 52. Scopus<br />35<br />
    53. 53. Citation overview from Scopus<br />The Citation Overview Results page lists documents that cited the document you selected from the Citation Overview.<br />You can include up to 5000 documents in a Citation Tracker. If you select more than 5000 documents only the first 5000 are included.<br />36<br />
    54. 54. Exercise - Article analysis<br />What is the most cited article written by Dr. Terry Delovitch? Tip: use both WOS and Scopus <br />37<br />
    55. 55. 38<br />
    56. 56. 39<br />
    57. 57. Publish or Perish <br />40<br />
    58. 58. Harzing's Publish or Perish <br /> Publish or Perish is a software program that retrieves and analyzes academic citations. It uses Google Scholar to obtain the raw citations, then analyzes these and presents the following statistics:<br /><ul><li>Total number of papers
    59. 59. Total number of citations
    60. 60. Average number of citations per paper
    61. 61. Average number of citations per author
    62. 62. Average number of papers per author
    63. 63. Average number of citations per year
    64. 64. h-index and related parameters
    65. 65. An analysis of the number of authors per paper.</li></ul>41<br />
    66. 66. Google Scholar vs. WOS dataset <br />Higher transparency <br />Coverage<br /><ul><li>Better coverage for unique citations
    67. 67. Not limited to journal articles published in ISI listed journals
    68. 68. Greater geographical coverage for non-US/UK based journals  
    69. 69. Better coverage of non-English sources </li></ul>Includes more comprehensive data for junior academics<br />All authors are listed<br />42<br />
    70. 70. Google Scholar vs. WOS dataset cont’<br />Accounts for minor citation errors <br />Does not account for duplications <br />Includes non-scholarly citations such as student handbooks, library guides or editorial notes.<br />Unclear coverage<br /><ul><li>uneven coverage across different fields of study
    71. 71. older materials are not well covered</li></ul>43<br />
    72. 72. Exercise - Article analysis<br />Find Bibliometricsinformation (# articles published, h-index, etc.) for Anne-WilHarzingusing the following: <br />WOS general search function: Author Finder <br />Scopus <br />Compare the results <br />44<br />
    73. 73. Download and install Publish or Perish at http://www.harzing.com/pop.htm#download<br />45<br />
    74. 74. F1000<br /><ul><li>Key metric: F1000 Article Factors (FFa)
    75. 75. Medicine and Biology specific </li></ul>46<br />
    76. 76. F1000 factors <br />The F1000 Article Factor (FFa) is calculated from the highest rating awarded by a Faculty Member (FM) plus an increment for each additional rating from other FMs.<br />For example, a single article that has been evaluated by three Faculty Members, who rated it ‘Recommended’, ‘Must Read’ and ‘Must Read’, will have its FFa calculated thus:<br />8 (highest rating ‘Must Read’) + 2 (increment for ‘Must Read’) + 1 (increment for ‘Recommended’) = 11.<br />47<br />
    77. 77. Which metric tool?<br />Adapted form https://dspace.ndlr.ie/jspui/handle/10633/27390<br />48<br />
    78. 78. Questions<br />Need library related assistance?Please don’t hesitate to contact us:<br />Nazi Torabi<br />519-661-2111 x88992 <br />ntorabi@uwo.ca<br />IM a librarian<br />https://www.lib.uwo.ca/chat/<br />49<br />

    ×