Successfully reported this slideshow.

Bibliometrics as evidence p. murphy hslg 2013


Published on

Published in: Health & Medicine, Technology
  • Be the first to comment

Bibliometrics as evidence p. murphy hslg 2013

  1. 1. Bibliometrics as EvidencePaul Murphy, RCSI Library pjmurphy@rcsi.ieA presentation to the Health Science Libraries Group Annual Conference, 11 April 2013
  2. 2. Royal College of Surgeons inIreland
  3. 3. Research evaluation contexts
  4. 4. Stakeholders and supportsCurrent ResearchInformation SystemResearch processmanagementReporting & analyticsBibliometricsInstitutionalrepositories
  5. 5. Royal College of Surgeons inIreland• Medicine, Pharmacy,Physiotherapy, Nursing,Faculties• Ireland, Bahrain, Malayasia• 1,850 u/grads, 365 p/grads,350 research staff• 5,935 publications, Web ofScience, 1996-• 535 publications in 2012 ofwhich 289 articlesSubject areas• Medicine General Internal 786• Psychiatry 537• Pharmacology 515• Surgery 473• Neurosciences 398• Biochemistry Molecular Biology374• Peripheral Vascular Disease 349• Clinical Neurology 293• Oncology 270• Obstetrics Gynecology 225
  6. 6. Individual profilingRCSI servicesInitiative: position librarians as experts in bibliometrics establish Library as stakeholder in research management consolidate repository and publication advisory support Initiated targeted personal profiling for Principal Investigators,senior academics and clinicians Demonstrated Departmental profiling for quality assurance,programme accreditation, institutional review Promoted self profiling, publications management for tenure andpromotion purposes with repository and EndNote informationskills sessionsInitiative: position librarians as experts in bibliometrics establish Library as stakeholder in research management consolidate repository and publication advisory support
  7. 7. Individual profiling - toolsCitation databases• Web of KnowledgeCitation Databases• Conference ProceedingsIndex• InCites• Scopus• Google Scholar( + Publish or Perish)Window services• Researcher ID• ORCID• ResearchGate• Google Scholar
  8. 8. Individual profiling –subscription databases
  9. 9. Link to repository
  10. 10. Google Scholar profile
  11. 11. Individual profiling tools:Google Scholar• Comparisons of citations in Web of Science, Scopus, andGoogle Scholar for articles published in general medicaljournals. JAMA. 2009 Sep 9;302(10):1092-6. doi: 10.1001/jama.2009.1307.• Comparison of PubMed and Google Scholar literature searches.Respir Care. 2010 May;55(5):578-83.• Medical literature searches: a comparison of PubMed andGoogle Scholar. Health Info Libr J. 2012 Sep;29(3):214-22. doi:10.1111/j.1471-1842.2012.00992.x.
  12. 12. Individual profiling tools: GoogleScholar + Publish or Perish
  13. 13. Individual profiling tools: GoogleScholar + Publish or Perish
  14. 14. Individual profiling tools: GoogleScholar + Publish or Perish
  15. 15. MyRI
  16. 16. MyRI
  17. 17. Institutional profilingWho? What? Where? When?• Who is an RCSI researcher?• Who was an RCSI researcher?• Specify inclusion / exclusioncriteria• Collaborator / co-authorshipguideline• Reckonable sites• Schedule of RCSI schools,academic, clinical, researchcentres• Peer reviewed journal papers?• Peer reviewed conferencepaper?• Meeting Abstract?• Report or specialist publication?• Review paper?• Meeting abstract?• Letter?• Outputs quantified by specificyears?• Which years?• Whole of life output or RCSIonly output?
  18. 18. Analysis – the metrics• Items published• By funding agency• By document type• By author ranked• By subject area ranked• By journal title ranked• By collaborating institutions /co-author addresses• By countries• By Items cited• By items not cited• Rankings by internal centre orlocation• By H-index
  19. 19. The Evidence – by times cited• Peripheral vascular disease 9,923• Psychiatry 9,094• Pharmacology & pharmacy 8,769• Neurosciences 7,217• Biochemistry & molecular biol5,888• Cardiac & cardiovascular 5,481• Hematology 4,273• Genetics & heredity 2,813• Oncology 2,757• Surgery 2,656• Clinical neurology 2,551• Cell biology 2,508• Microbiology 2,210• Immunology 1,844• Public, environmental &occupational health 1,811• Endocrinology & metabolism1,753• Biochem research methods 1,587• Nutrition & dietetics 1,557• Medicine, general & internal 1,551• Obstetrics & gynecology 1,445• Pathology 1,441• Infectious diseases 1,389• Biophysics 1,321• Psychology 1,250• Respiratory system 1,127
  20. 20. Sourcing & quantificationSources• Research database – CurrentResearch Information System• PubMed• Embase• Web of Science CitationIndexes• PsychInfo• CINAHL• Agency publisher: DoH, WHO• Institutional repositoriesItems per sourceValidation and normalisation• Personal name variants• Institutional name variantsProcessing: time, resources• Who?• What? EndNote, CRIS?
  21. 21. Retrieval and processingRCSI affiliatedSource Time Results: allitems, all typesAll items, year2010Web of Science 1945- 4,879 492PubMed 1950s- 2,034 254Embase and Medline combined 1974- 3,904 538CINAHL 1937- 265 53Records to validate and de-duplicate 11,082 1,337Estimated records annually from 2010- annual 350
  22. 22.
  23. 23. SCImago - W Eur - Medicine
  24. 24. Institutional metrics• Spend on teaching• Graduation time/ rates• Progression / successrates• Student:staff ratio• Student profile• Student satisfaction• Programmedevelopment• Spend on research• Research income• Publication output &impact• Industry linkage• Doctoral productivity• Knowledge exchange• Licences / patents• International staff
  25. 25. 2007
  26. 26. p82
  27. 27. p83
  28. 28. Institutional profiling -comparative• Using the TR database Essential Science Indicators ESI• ESI uses different subject categories to Web of Science• Only valid comparator subject category = “Clinical Medicine”
  29. 29. Essential Science Indicators2002-2012Rank from 3,739in ClinicalMedicineInstitution name in database Papers Citations Citations perpaper355 TRINITY COLL DUBLIN 2,731 42,479 15.55366 UNIV COLL DUBLIN 2,699 41,320 15.31658 NATL UNIV IRELAND UNIV COLL CORK 1,428 18,985 13.29779 ROYAL COLL SURGEONS IRELAND 816 14,868 18.221,241 NATL UNIV IRELAND GALWAY 594 7,716 12.992,267 DUBLIN CITY UNIV 272 3,133 11.523,367 UNIV LIMERICK 274 1,827 6.67Harvard Johns Hopkins Mayo Clin U Calif SFU Washington U Calif LA U Michigan U Toronto U Penn
  30. 30. Institutional profiling –comparative and analyticNational Reseach Data Project• Co-ordinated by Director of Research, Irish UniversitiesAssociation• Higher Education Authority funded to 2009 -2012 nowinstitutionally subscribed• UCC UL UCD NUIG NUIM TCD DIT RCSI• Creating a national citation dataset 1980-• TR InCites engine• Data quality: normalization and validation• Analytics to informing national research prioritization exercisesheld by Higher Education Authority, Science Foundation Ireland,Office of Science, Technology and Innovation
  31. 31. National Research Data ProjectOutcomes and targets1. To employ locally the resources and metrics which are used toevaluate our universities internationally2. To provide a reliable source of quality-controlled hard data uponwhich to base our research analyses3. To map and analyse collaborations (based on author affiliationsand addresses associated with individual publications)4. To produce ready data for research audits and researchevaluations to support the emerging cross-institutional researchand graduate school collaborations
  32. 32. National Research Data ProjectNational Citation Report Ireland dataset at Feb 2013Contains 201,471 source articles from 209,842 authors publishedbetween 1979 and 2013.Also contains 1,487,103 articles that have cited the 201,471source articles in this dataset.
  33. 33. InCites
  34. 34. InCites
  35. 35. RCSI InCites NRDP
  36. 36. Global comparisonIreland - Medicine
  37. 37. InCites ComparisonIreland and EU 15 Medicine General
  38. 38. Altmetrics• Altmetrics (alternate or alternative metrics) isthe study of newer web-based metrics inscholarship that look at total influence ofsomeone via social media.• Priem J, Taraborelli D, Groth P, Neylon C. Alt-metrics: amanifesto. 26 October 2010. App for Scopus a basic framework for webometricsL Björneborn, P Ingwersen – JASIST, 2004We define webometrics within the framework of informetricstudies and bibliometrics as belonging to library andinformation science.Cited by 224 Related articles
  39. 39. Conclusions• Librarians established as research support stakeholders• Librarians established as experts on bibliometrics• Bibliometrics, especially citation distribution, are validatedmeasures of research evaluation and performance• Bibliometrics to be reported and incorporated into CurrentResearch Information System• Bibliometrics an element in comparative and benchmarkingassessments using multiple dimensions: geographic, researchfront, expected relative performance indicators.• Subscription products are the core tools but free resources andalternative resources merit further investigation
  40. 40. Bibliometrics as EvidencePaul Murphy, RCSI Library pjmurphy@rcsi.ieA presentation to the Health Science Libraries Group Annual Conference, 11 April 2013