One Entry to Research: critical assessment of Web of Sceince, Scopus and Google Scholar.Presentation Transcript
One Entry to Research: critical assessment of Web of Science(WoS), Scopus and Google Scholar(GS) 10.20-11.00 Thursday 31 August BIBSAM workshop, National Library, Sthlm Speaker: Lars Iselid
BIBSAM-project from 1/1 2006 – 31/8 2006: One Entry to Research – critical assessment of Web of Science, Scopus och Google Scholar
Evaluations of multidisciplinary citation databases
Address and department/institution search
Subject search and coverage
Overlap and coverage between Scopus, WoS etc (Not GS)
Questions for the libraries?
Shall we keep Web of Science or supplement with or change to Scopus?
Or do we need them at all when we have Google Scholar for free?
Is it sufficient to search for citations in Web of Science or should we also consider Scopus and Google Scholar?
Are these databases fullworthy as subject databases?
Answers from a librarian!
No, we should not use Google Scholar exclusively, which has lots of flaws for both citation and subject search.
If we want to find more cited references concerning research from 1996 and present we have to consider Scopus (and Google Scholar even pre-1996) beside of Web of Science.
Google Scholar has often unique citations in comparision with Scopus and Web of Science.
Rantapaa* 8 name variants
Rantapaa S* 2 additional name variants Of course much of the incorrect author spellings in Cited Ref Search could be blamed on incorrect citation information from article authors.
The Lancet-article? Rantapaa Dahlqvist or Rantapaa-Dahlqvist
The Lancet-article? Rantapaadahlqvist
The Lancet-article? Rantapaa-Dahlquist
The Lancet article! Original e-journal article
Lancet article in
Lancet article in Dahlqvist, S.R.
Lancet article in
Due to all incorrect information in author and address fields I recommend to compare with a publication list from the researcher. Publication lists
Author search for finding times cited
AG Fisher’s affiliation is Umeå but has been in Colorado before.
5 AG Fisher with three first name initials
Has AG Fisher published articles as A Fisher? 8 hits but no one by Anne G Fisher
Two articles with spelling A Fisher with search limits Am J Occup Ther 1987 (where address is not Umea or Colorado)
Two AG Fisher has published in same journal. But none with address Colorado or Umea. Right Anne Fisher! False Anne Fisher!
Is it of great value to compare with publication lists?
Yes, too often.
Do researchers really manage to keep track of their articles?
No, many of them don’t. But we must tell them to do that if they want to hurry up their career.
One of the articles by PO Sandman does not exist in his publication list. It’s cited 2 times in WoS.
These articles from WoS are not existing in the publication list of S Eriksson. 14 times cited.
Bad implementation of data! Address is missing for Rantapää
… but in the original e-journal article, it’s there!
Search with limit umea could produce false amount of citations!! S Astrom umea
S Astrom Dept ophthalmol
Eriksson s* umea
S Eriksson Umea plant sci ctr
S Eriksson Geriatr Ctr
Staffan Eriksson at the same department published as S Eriksson
Variants of a department
Dept Community Med & Rehabil
Inst Community Med & Rehabil, Div Geriatr
Dept Geriatr Med
Lost citations when using search limit author and address.
Author-address search solutions from Thomson ISI WoS
$ equals one or zero characters
* any number of characters
? is one single characters
Author=astrom s Address=umea SAME dept nursing
Retrieves at same line: umea univ dept nursing
Lund univ, Dept nursing
Umea univ, Dept Ophthalmology
Lots of flaws when counting citations in Google Scholar. Many duplicates! Has no address field.
Why all these errors?
Lacking (raw)data from the beginning.
Unsuccessful indexing of (raw)data from the vendors Elsevier, Thomson och Google.
Algorithms can’t solve lacking (raw)data, perhaps tune up some errors.
It’s not a question about algo’s, it’s a question about structured well-indexed data.
Multidisciplinary subject search
KeyWord Plus in ” Key Words Plus supplies additional search terms extracted from the titles of articles cited by authors in their bibliographies and footnotes”. ” ,,,provide a brief condensation of the major and minor themes Discussed”. Källa: www.garfield.library.upenn.edu/essays/v13p295y1990.pdf Records without references won't have KW+ - but more specifically, articles whose references are not linked to source items. In addition, it may be that those with very few linked references won't generate good candidates for KW+ either.
Hormone replacement therapy
Hormone replacement therapy in cited references of the article.
29% 37,7% 17% 16% 0,3%
Subject words (and reference information) are
collected from Elsevier databases och Medline:
Geobase subject index
Emtree - bara major headings are displyed!
MeSH – bara major headings are displayed!
FLX terms, WTA terms
Author keywords (uncontrolled)
Check Major MeSH Headings in a PubMed reference
Breast Neoplasms/surgery* is missing!
Emtree terms in Embase
Same reference in Scopus misses Emtree terms
Same thing with Compendex
… and same reference in Scopus
Lacking subject search
No index search to get suggestions of terms.
Thesauri are inconsistently integrated, sometimes no or just a selected part of (major) MeSH-terms, Emtree and Compendex terms.
Uncontrolled terms mixed with controlled terms. No possibility to limit to controlled terms exclusive.
No option for chosing thesauri.
No mapping of terms, similar to PubMed and Embase.
No possibility to explode a search for subterms.
No integration of MeSH subheadings.
Subject search in Google Scholar advanced search
Subject coverage in
Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Google Scholar: An Empirical Study Portal: Libraries and the Academy Vol. 6, No. 2, pp. 127-141.
From 47 databases 50 article titles were randomly collected from each database and compared with Google Scholar.
Subject coverage Källa: Neuhaus, Chris et al (2006)
Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 60%.
Coverage evaluation of WoS and Scopus accomplished by Ylva Gavel, KIB, och Lars Iselid, Umeå UB.
We’ve been matching ISSN against Ulrich’s journal database to exclude non-active titles, obsolete/invalid ISSN, titles not covered. It’s remarkably many.
Study will be published in a scientific journal if accepted.
Coverage evaluation of WoS and Scopus - How many active titles? Scopus 13.226 journals WoS 8.786 journals
Coverage evaluation of WoS and Scopus - How many active titles overlap and how many are unique? WoS Scopus 7.210 indexed both in Scopus and WoS All together 14.802 journals with unique ISSN.
Coverage evaluation of WoS and Scopus - How many active titles overlap? WoS covers 55% of Scopus Scopus covers 82% of WoS 49 % exist in both
Active titles in each database
Average overlap of WoS 51,8% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 55% 55% 46% 57% 36% 62%
Average overlap of Scopus 74,2% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 82% 91% 45% 71% 67% 89%
Coverage evaluation Scopus and Medline - How many active titles? Scopus 13.226 journals Medline 4.843 journals
Coverage evaluation Scopus and Medline - How many active titles is overlapping and how many unique? Medline Scopus 4.425 are indexed by both Scopus and Medline All together 13.644 journals with unique ISSN
Coverage evaluation Scopus and Medline - How many journals overlap? Medline covers 33% of Scopus Scopus covers 91% of Medline 32 % exist in both
About one third of all unique journals in Scopus and Medline are indexed in both databases.
Coverage evaluation WoS vs. Medline - How many active titles? WoS 8.786 journals Medline 4.843 journals
Coverage evaluation WoS vs. Medline - How many active titles is overlapping and how many unique? Medline WoS 3.001 indexed By both WoS and Medline All together 10.682 journals with unique ISSN.
Coverage evaluation WoS vs. Medline - How many journals overlap? Medline covers 34% of WoS WoS covers 62% of Medline 28 % exist in both
Nearly one third of all unique journals in WoS and Medline are indexed in both databases.
New options for author search: Scopus Author Identifier, WoS Author Finder Some improvements, some confusions!
Displays not just the address of the author retrieved, but also all co-authors addresses. Just displays institution, not department. Author Finder
Refine your results
What to do about the mess?
Discuss the value of a citation and discuss alternative methods for measuring research.
Discuss and measure the errors in the databases concerning citation search.
By continuously evaluating citation databases we can put som pressure on the vendors to work more with improvements of the flaws.