Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
—
•
•
•
•
•
AGENDA
— BACKGROUND
•
•
—
TRENDS IN SCHOLARLY RESEARCH
—
—
—
—
—
WHY DO INSTITUTIONS EVALUATE RESEARCH?
—
—
—
—
—
WHY DO INSTITUTIONS EVALUATE RESEARCH?
WHY USE BIBLIOMETRICS?
STAKEHOLDERS IN RESEARCH EVALUATION
WHAT DO CITATION COUNTS REALLY MEASURE?
WHAT DO CITATION COUNTS REALLY MEASURE?
—
BIBLIOMETRIC PERSPECTIVES ON SELF-CITATION
•
•
•
•
̶
̶
̶
—
USEFULNESS ≠ QUALITY
CITATION FREQUENCY CORRELATES WITH
OTHER MEASURES OF PEER ESTEEM
BEST PRACTICE: INFORMED PEER REVIEW
CITATION METRICS ARE ONE PIECE OF
THE RESEARCH PERFORMANCE PUZZLE.
—
—
—
•
•
•
•
•
—
•
•
— WEB OF SCIENCE
CITATION METRICS
1
•
•
CITATION METRICS ARE ONLY AS GOOD AS THEIR SOURCE
THE WEB OF SCIENCE CORE COLLECTION
SOURCE & FOUNDATION
̶
̶
̶
— APPROPRIATE USE
•
•
SAN FRANCISCO DECLARATION ON
RESEARCH ASSESSMENT (DORA)
—
THE LEIDEN MANIFESTO FOR RESEARCH METRICS
—
EXERCISES USING INCITES
•
•
•
USE A VARIETY OF INDICATORS
ADDITIONAL REFERENCE MATERIAL
— JOURNAL ANALYSIS
•
•
—
JOURNAL IMPACT FACTOR REFLECTS
A JOURNAL’S OVERALL PERFORMANCE
—
Eigenfactor score reflects a journal’s footprint in the overall journal-citation
network, measuring its influence in the...
— NORMALIZED EIGENFACTOR SCORE
Normalized Eigenfactor
Score: a value of 1 indicates
average influence. A higher
value indi...
— INSTITUTIONAL ANALYSIS
3
•
WHY NORMALIZE?
0
500
1000
1500
2000
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87
...
36
ACCOUNT FOR FIELD, AGE, AND DOCUMENT TYPE
IS 20 CITATIONS GOOD OR BAD?
ACCOUNT FOR FIELD, AGE, AND DOCUMENT TYPE
Percentile in subject area
smaller is better — in this example 0.04% of the papers in the
category (plant sciences) in 201...
— AUTHOR ANALYSIS
3
•
AUTHOR IDENTIFICATION
40
Author clustering
-applied to Web of Science
Core Collection regularly
6.2 million Web
of Science...
THE H-INDEX
DETERMINING H-INDEX
—
—
•
•
—
•
—
USING H-INDEX IN RESEARCH EVALUATION
THESE THREE AUTHORS HAVE THE SAME H-INDEX (4)
THANK YOU
— APPENDIX
4
—
WHY CITATION METRICS?
—
—
—
—
Bornmann, L., & Marx, W. (2015). Methods for the generation of normalized citation impact scores in bibliometrics: Which m...
THOMSON REUTERS – THE AUTHORITY ON CITATION DATA
KEEP DATA COLLECTION AND ANALYTICAL PROCESSES
OPEN, TRANSPARENT, AND SIMPLE
ACCOUNT FOR VARIATION BY FIELD IN PUBLICATION AND
CITATION PRACTICES
Understand the subject areas you are using
InCites su...
BIBLIOMETRIC PERSPECTIVES ON “NEGATIVE” CITATION
citations
citation stacking
article-type manipulation
self-citation false publications
mentions
Purchased likes mention bo...
Upcoming SlideShare
Loading in …5
×

Using Bibliometrics in the Library

515 views

Published on

Understand the foundations of bibliometrics, how these methods can be applied, and best practices for using these techniques.

Published in: Education
  • Be the first to comment

Using Bibliometrics in the Library

  1. 1. —
  2. 2. • • • • • AGENDA
  3. 3. — BACKGROUND • •
  4. 4. — TRENDS IN SCHOLARLY RESEARCH
  5. 5. — — — — — WHY DO INSTITUTIONS EVALUATE RESEARCH?
  6. 6. — — — — — WHY DO INSTITUTIONS EVALUATE RESEARCH?
  7. 7. WHY USE BIBLIOMETRICS?
  8. 8. STAKEHOLDERS IN RESEARCH EVALUATION
  9. 9. WHAT DO CITATION COUNTS REALLY MEASURE?
  10. 10. WHAT DO CITATION COUNTS REALLY MEASURE?
  11. 11.
  12. 12. BIBLIOMETRIC PERSPECTIVES ON SELF-CITATION • • • • ̶ ̶ ̶
  13. 13.
  14. 14. USEFULNESS ≠ QUALITY
  15. 15. CITATION FREQUENCY CORRELATES WITH OTHER MEASURES OF PEER ESTEEM
  16. 16. BEST PRACTICE: INFORMED PEER REVIEW
  17. 17. CITATION METRICS ARE ONE PIECE OF THE RESEARCH PERFORMANCE PUZZLE. — — — • • • • • — • •
  18. 18. — WEB OF SCIENCE CITATION METRICS 1 • •
  19. 19. CITATION METRICS ARE ONLY AS GOOD AS THEIR SOURCE
  20. 20. THE WEB OF SCIENCE CORE COLLECTION
  21. 21. SOURCE & FOUNDATION ̶ ̶ ̶
  22. 22. — APPROPRIATE USE • •
  23. 23. SAN FRANCISCO DECLARATION ON RESEARCH ASSESSMENT (DORA)
  24. 24.
  25. 25. THE LEIDEN MANIFESTO FOR RESEARCH METRICS
  26. 26. — EXERCISES USING INCITES • • •
  27. 27. USE A VARIETY OF INDICATORS
  28. 28. ADDITIONAL REFERENCE MATERIAL
  29. 29. — JOURNAL ANALYSIS • •
  30. 30.
  31. 31. JOURNAL IMPACT FACTOR REFLECTS A JOURNAL’S OVERALL PERFORMANCE
  32. 32. — Eigenfactor score reflects a journal’s footprint in the overall journal-citation network, measuring its influence in the entire network. It is based on the Google PageRank method. Journal C has a higher PageRank or “weight” than Journal E, even though it has fewer links citing it; the one cite it does have is of a much higher value. From Wikipedia – “PageRank” EIGENFACTOR SCORE
  33. 33. — NORMALIZED EIGENFACTOR SCORE Normalized Eigenfactor Score: a value of 1 indicates average influence. A higher value indicates above average influence
  34. 34. — INSTITUTIONAL ANALYSIS 3 •
  35. 35. WHY NORMALIZE? 0 500 1000 1500 2000 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 0 500 1000 1500 2000 0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72 75 78 81 84 87 10 7.5
  36. 36. 36 ACCOUNT FOR FIELD, AGE, AND DOCUMENT TYPE
  37. 37. IS 20 CITATIONS GOOD OR BAD? ACCOUNT FOR FIELD, AGE, AND DOCUMENT TYPE
  38. 38. Percentile in subject area smaller is better — in this example 0.04% of the papers in the category (plant sciences) in 2014 had more citations ACCOUNT FOR FIELD, AGE, AND DOCUMENT TYPE Average percentile for a group of papers, we average all of the documents’ percentiles
  39. 39. — AUTHOR ANALYSIS 3 •
  40. 40. AUTHOR IDENTIFICATION 40 Author clustering -applied to Web of Science Core Collection regularly 6.2 million Web of Science Core Collection records •722,000 ResearcherIDs •7.7 million Web of Science Core Collection records AUTOMATED AUTHOR VERIFIED
  41. 41. THE H-INDEX
  42. 42. DETERMINING H-INDEX
  43. 43. — — • • — • — USING H-INDEX IN RESEARCH EVALUATION
  44. 44. THESE THREE AUTHORS HAVE THE SAME H-INDEX (4)
  45. 45. THANK YOU
  46. 46. — APPENDIX 4
  47. 47.
  48. 48. WHY CITATION METRICS? — — —
  49. 49.
  50. 50. Bornmann, L., & Marx, W. (2015). Methods for the generation of normalized citation impact scores in bibliometrics: Which method best reflects the judgements of experts? Journal of Informetrics, 9(2), 408-418. doi:10.1016/j.joi.2015.01.006 Clark, K.E. (1957). The APA study of psychologists. American Psychologist, 9, 117–120. Cole, S., and Cole, J.R. (1967). Scientific output and recognition: A study in the operation of the reward system in science. American Sociological Review, 32, 377–390. Derrick, G. E., Haynes, A., Chapman, S., & Hall, W. D. (2011). The Association between Four Citation Metrics and Peer Rankings of Research Influence of Australian Researchers in Six Fields of Public Health. Plos One, 6(4). doi:10.1371/journal.pone.0018521 Garfield, E., and Welljams-Dorof, A. (1992a). Of Nobel class: A citation perspective on high impact research authors. Theoretical Medicine, 13, 118–126. Lovegrove, B. G., & Johnson, S. D. (2008). Assessment of research performance in biology: How well do peer review and bibliometry correlate? Bioscience, 58(2), 160-164. doi:10.1641/b580210 Mryglod, O., Kenna, R., Holovatch, Y., & Berche, B. (2013). Absolute and specific measures of research group excellence. Scientometrics, 95(1), 115-127. doi:10.1007/s11192-012- 0874-7 Norris, M., & Oppenheim, C. (2010). Peer review and the h-index: Two studies. Journal of Informetrics, 4(3), 221-232. doi:10.1016/j.joi.2009.11.001 Oppenheim, C. (1997). The correlation between citation counts and the 1992 research assessment exercise ratings for British research in genetics, anatomy and archaeology. Journal of Documentation, 53(5), 477-487. doi:10.1108/eum0000000007207 Small, H.G. (1977). Co-citation model of a scientific specialty: – a longitudinal study of collagen research. Social Studies of Science, 7 (2), 139–166. Smith, A.T., and Eysenck, M. (2002). The correlation between RAE rankings and citation counts in psychology. Technical Report, Psychology, University of London, Royal Holloway. http://cogprints.ecs.soton.ac.uk/archive/00002749/01/citations.pdf Van Raan, A. F. J. (2006). Comparison of the Hirsch-index with standard bibliometric indicators and with peer judgment for 147 chemistry research groups. Scientometrics, 67(3), 491- 502. doi:10.1556/Scient.67.2006.3.10 Vieira, E. S., Cabral, J. A. S., & Gomes, J. (2014). How good is a model based on bibliometric indicators in predicting the final decisions made by peers? Journal of Informetrics, 8(2), 390-405. doi:10.1016/j.joi.2014.01.012 Virgo, J. A. (1977). A Statistical Procedure for Evaluating the Importance of Scientific Papers. Library Quarterly, 47 (4), 415-430. VALIDATION STUDY BIBLIOGRAPHY
  51. 51. THOMSON REUTERS – THE AUTHORITY ON CITATION DATA
  52. 52. KEEP DATA COLLECTION AND ANALYTICAL PROCESSES OPEN, TRANSPARENT, AND SIMPLE
  53. 53. ACCOUNT FOR VARIATION BY FIELD IN PUBLICATION AND CITATION PRACTICES Understand the subject areas you are using InCites subject area schemes Source Type Web of Science Thomson Reuters Journal to category Essential Science Indicators Thomson Reuters Journal to category Global Institutional Profiles Project (GIPP) Thomson Reuters Category to category ANVUR Italy Category to category Australia ERA Australia Journal to category China SCACD China Category to category FAPESP Brazil Category to category OECD (Frascati) OECD Category to category UK RAE and REF UK Category to category KAKEN Japan Category to category
  54. 54. BIBLIOMETRIC PERSPECTIVES ON “NEGATIVE” CITATION
  55. 55. citations citation stacking article-type manipulation self-citation false publications mentions Purchased likes mention bots Purchased retweets social media promotion tools & Bots Usage Bots Harvesters & scrapers

×