One Entry to Research: critical assessment of Web of Sceince, Scopus and Google Scholar.

Uploaded on


  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Excellent. You've shown your credibility on presentation with this slideshow. This one deserves thumbs up. I'm John, owner of . Perhaps I'll get to see more quality slides from you.

    Best wishes.
    Are you sure you want to
    Your message goes here
  • Outstanding display. Really clear together with helpful

    Are you sure you want to
    Your message goes here
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. One Entry to Research: critical assessment of Web of Science(WoS), Scopus and Google Scholar(GS) 10.20-11.00 Thursday 31 August BIBSAM workshop, National Library, Sthlm Speaker: Lars Iselid
  • 2. BIBSAM-project from 1/1 2006 – 31/8 2006: One Entry to Research – critical assessment of Web of Science, Scopus och Google Scholar
  • 3. Evaluations of multidisciplinary citation databases
    • Citation search
      • Author search
      • Address and department/institution search
    • Subject search and coverage
    • Overlap and coverage between Scopus, WoS etc (Not GS)
  • 4. Questions for the libraries?
    • Shall we keep Web of Science or supplement with or change to Scopus?
    • Or do we need them at all when we have Google Scholar for free?
    • Is it sufficient to search for citations in Web of Science or should we also consider Scopus and Google Scholar?
    • Are these databases fullworthy as subject databases?
  • 5. Answers from a librarian!
    • No, we should not use Google Scholar exclusively, which has lots of flaws for both citation and subject search.
    • If we want to find more cited references concerning research from 1996 and present we have to consider Scopus (and Google Scholar even pre-1996) beside of Web of Science.
    • Google Scholar has often unique citations in comparision with Scopus and Web of Science.
  • 6. Rantapaa* 8 name variants
  • 7. Rantapaa S* 2 additional name variants Of course much of the incorrect author spellings in Cited Ref Search could be blamed on incorrect citation information from article authors.
  • 8. The Lancet-article? Rantapaa Dahlqvist or Rantapaa-Dahlqvist
  • 9. The Lancet-article? Rantapaadahlqvist
  • 10. The Lancet-article? Rantapaa-Dahlquist
  • 11. The Lancet article! Original e-journal article
  • 12. Lancet article in
  • 13. Rantapaa* S
  • 14. Lancet article in Dahlqvist, S.R.
  • 15. Lancet article in
  • 16. Due to all incorrect information in author and address fields I recommend to compare with a publication list from the researcher. Publication lists
  • 17. Author search for finding times cited
    • AG Fisher’s affiliation is Umeå but has been in Colorado before.
    • 5 AG Fisher with three first name initials
  • 18. Has AG Fisher published articles as A Fisher? 8 hits but no one by Anne G Fisher
  • 19. Two articles with spelling A Fisher with search limits Am J Occup Ther 1987 (where address is not Umea or Colorado)
  • 20. Two AG Fisher has published in same journal. But none with address Colorado or Umea. Right Anne Fisher! False Anne Fisher!
  • 21.
    • Is it of great value to compare with publication lists?
    • Yes, too often.
    • Do researchers really manage to keep track of their articles?
    • No, many of them don’t. But we must tell them to do that if they want to hurry up their career.
  • 22. One of the articles by PO Sandman does not exist in his publication list. It’s cited 2 times in WoS.
  • 23. These articles from WoS are not existing in the publication list of S Eriksson. 14 times cited.
  • 24. Bad implementation of data! Address is missing for Rantapää
  • 25. … but in the original e-journal article, it’s there!
  • 26. Search with limit umea could produce false amount of citations!! S Astrom umea
  • 27. S Astrom Dept ophthalmol
  • 28. Eriksson s* umea
  • 29. S Eriksson Umea plant sci ctr
  • 30. S Eriksson Geriatr Ctr
  • 31. Staffan Eriksson at the same department published as S Eriksson
  • 32. Variants of a department
    • Dept Community Med & Rehabil
    • Inst Community Med & Rehabil, Div Geriatr
    • Dept Geriatr Med
    • Geriatr ctr
  • 33. Lost citations when using search limit author and address.
  • 34. Author-address search solutions from Thomson ISI WoS
    • Rantapaa$dahlq?ist s*
      • $ equals one or zero characters
      • * any number of characters
      • ? is one single characters
    • Author=astrom s Address=umea SAME dept nursing
      • Retrieves at same line: umea univ dept nursing
    • Doesn’t retrieve:
      • Lund univ, Dept nursing
      • Umea univ, Dept Ophthalmology
  • 35. Lots of flaws when counting citations in Google Scholar. Many duplicates! Has no address field.
  • 36. Why all these errors?
    • Lacking (raw)data from the beginning.
    • Unsuccessful indexing of (raw)data from the vendors Elsevier, Thomson och Google.
    • Algorithms can’t solve lacking (raw)data, perhaps tune up some errors.
    • It’s not a question about algo’s, it’s a question about structured well-indexed data.
  • 37. Multidisciplinary subject search
  • 38. KeyWord Plus in ” Key Words Plus supplies additional search terms extracted from the titles of articles cited by authors in their bibliographies and footnotes”. ” ,,,provide a brief condensation of the major and minor themes Discussed”. Källa: Records without references won't have KW+ - but more specifically, articles whose references are not linked to source items. In addition, it may be that those with very few linked references won't generate good candidates for KW+ either.
  • 39. Hormone replacement therapy
  • 40. Hormone replacement therapy in cited references of the article.
  • 41. 29% 37,7% 17% 16% 0,3%
  • 42. Keywords/Thesaurus
    • Subject words (and reference information) are
    • collected from Elsevier databases och Medline:
      • Geobase subject index
      • Emtree - bara major headings are displyed!
      • MeSH – bara major headings are displayed!
      • FLX terms, WTA terms
      • Regional Index
      • Species Index
      • Ei thesaurus
      • Author keywords (uncontrolled)
  • 43. Check Major MeSH Headings in a PubMed reference
  • 44. Breast Neoplasms/surgery* is missing!
  • 45. Emtree terms in Embase
  • 46. Same reference in Scopus misses Emtree terms
  • 47. Same thing with Compendex
  • 48. … and same reference in Scopus
  • 49. Lacking subject search
    • No index search to get suggestions of terms.
    • Thesauri are inconsistently integrated, sometimes no or just a selected part of (major) MeSH-terms, Emtree and Compendex terms.
    • Uncontrolled terms mixed with controlled terms. No possibility to limit to controlled terms exclusive.
    • No option for chosing thesauri.
    • No mapping of terms, similar to PubMed and Embase.
    • No possibility to explode a search for subterms.
    • No integration of MeSH subheadings.
  • 50. Subject search in Google Scholar advanced search
  • 51. Subject coverage in
    • Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Google Scholar: An Empirical Study Portal: Libraries and the Academy Vol. 6, No. 2, pp. 127-141.
    • From 47 databases 50 article titles were randomly collected from each database and compared with Google Scholar.
  • 52. Subject coverage Källa: Neuhaus, Chris et al (2006)
  • 53. Database coverage
    • Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 60%.
  • 54. Coverage evaluation of WoS and Scopus accomplished by Ylva Gavel, KIB, och Lars Iselid, Umeå UB.
    • We’ve been matching ISSN against Ulrich’s journal database to exclude non-active titles, obsolete/invalid ISSN, titles not covered. It’s remarkably many.
    • Study will be published in a scientific journal if accepted.
  • 55. Coverage evaluation of WoS and Scopus - How many active titles? Scopus 13.226 journals WoS 8.786 journals
  • 56. Coverage evaluation of WoS and Scopus - How many active titles overlap and how many are unique? WoS Scopus 7.210 indexed both in Scopus and WoS All together 14.802 journals with unique ISSN.
  • 57. Coverage evaluation of WoS and Scopus - How many active titles overlap? WoS covers 55% of Scopus Scopus covers 82% of WoS 49 % exist in both
  • 58. Active titles in each database
  • 59. Average overlap of WoS 51,8% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 55% 55% 46% 57% 36% 62%
  • 60. Average overlap of Scopus 74,2% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 82% 91% 45% 71% 67% 89%
  • 61. Coverage evaluation Scopus and Medline - How many active titles? Scopus 13.226 journals Medline 4.843 journals
  • 62. Coverage evaluation Scopus and Medline - How many active titles is overlapping and how many unique? Medline Scopus 4.425 are indexed by both Scopus and Medline All together 13.644 journals with unique ISSN
  • 63. Coverage evaluation Scopus and Medline - How many journals overlap? Medline covers 33% of Scopus Scopus covers 91% of Medline 32 % exist in both
    • About one third of all unique journals in Scopus and Medline are indexed in both databases.
  • 64. Coverage evaluation WoS vs. Medline - How many active titles? WoS 8.786 journals Medline 4.843 journals
  • 65. Coverage evaluation WoS vs. Medline - How many active titles is overlapping and how many unique? Medline WoS 3.001 indexed By both WoS and Medline All together 10.682 journals with unique ISSN.
  • 66. Coverage evaluation WoS vs. Medline - How many journals overlap? Medline covers 34% of WoS WoS covers 62% of Medline 28 % exist in both
    • Nearly one third of all unique journals in WoS and Medline are indexed in both databases.
  • 67. New options for author search: Scopus Author Identifier, WoS Author Finder Some improvements, some confusions!
  • 68. Displays not just the address of the author retrieved, but also all co-authors addresses. Just displays institution, not department. Author Finder
  • 69. Refine your results
  • 70. What to do about the mess?
    • Discuss the value of a citation and discuss alternative methods for measuring research.
    • Discuss and measure the errors in the databases concerning citation search.
    • By continuously evaluating citation databases we can put som pressure on the vendors to work more with improvements of the flaws.