One Entry to Research: critical assessment of Web of Science(WoS), Scopus and Google Scholar(GS) 10.20-11.00 Thursday 31 August BIBSAM workshop, National Library, Sthlm Speaker: Lars Iselid

BIBSAM-project from 1/1 2006 – 31/8 2006: One Entry to Research – critical assessment of Web of Science, Scopus och Google Scholar

Evaluations of multidisciplinary citation databases Citation search Author search Address and department/institution search Subject search and coverage Overlap and coverage between Scopus, WoS etc (Not GS)

Citation search

Author search

Address and department/institution search

Subject search and coverage

Overlap and coverage between Scopus, WoS etc (Not GS)

Questions for the libraries? Shall we keep Web of Science or supplement with or change to Scopus? Or do we need them at all when we have Google Scholar for free? Is it sufficient to search for citations in Web of Science or should we also consider Scopus and Google Scholar? Are these databases fullworthy as subject databases?

Shall we keep Web of Science or supplement with or change to Scopus?

Or do we need them at all when we have Google Scholar for free?

Is it sufficient to search for citations in Web of Science or should we also consider Scopus and Google Scholar?

Are these databases fullworthy as subject databases?

Answers from a librarian! No, we should not use Google Scholar exclusively, which has lots of flaws for both citation and subject search. If we want to find more cited references concerning research from 1996 and present we have to consider Scopus (and Google Scholar even pre-1996) beside of Web of Science. Google Scholar has often unique citations in comparision with Scopus and Web of Science.

No, we should not use Google Scholar exclusively, which has lots of flaws for both citation and subject search.

If we want to find more cited references concerning research from 1996 and present we have to consider Scopus (and Google Scholar even pre-1996) beside of Web of Science.

Google Scholar has often unique citations in comparision with Scopus and Web of Science.

Rantapaa* 8 name variants

Rantapaa S* 2 additional name variants Of course much of the incorrect author spellings in Cited Ref Search could be blamed on incorrect citation information from article authors.

The Lancet-article? Rantapaa Dahlqvist or Rantapaa-Dahlqvist

The Lancet-article? Rantapaadahlqvist

The Lancet-article? Rantapaa-Dahlquist

The Lancet article! Original e-journal article

Lancet article in

Rantapaa* S

Lancet article in Dahlqvist, S.R.

Lancet article in

Due to all incorrect information in author and address fields I recommend to compare with a publication list from the researcher. Publication lists

Author search for finding times cited AG Fisher’s affiliation is Umeå but has been in Colorado before. 5 AG Fisher with three first name initials

AG Fisher’s affiliation is Umeå but has been in Colorado before.

5 AG Fisher with three first name initials

Has AG Fisher published articles as A Fisher? 8 hits but no one by Anne G Fisher

Two articles with spelling A Fisher with search limits Am J Occup Ther 1987 (where address is not Umea or Colorado)

Two AG Fisher has published in same journal. But none with address Colorado or Umea. Right Anne Fisher! False Anne Fisher!

Is it of great value to compare with publication lists? Yes, too often. Do researchers really manage to keep track of their articles? No, many of them don’t. But we must tell them to do that if they want to hurry up their career.

Is it of great value to compare with publication lists?

Yes, too often.

Do researchers really manage to keep track of their articles?

No, many of them don’t. But we must tell them to do that if they want to hurry up their career.

One of the articles by PO Sandman does not exist in his publication list. It’s cited 2 times in WoS.

These articles from WoS are not existing in the publication list of S Eriksson. 14 times cited.

Bad implementation of data! Address is missing for Rantapää

… but in the original e-journal article, it’s there!

Search with limit umea could produce false amount of citations!! S Astrom umea

S Astrom Dept ophthalmol

Eriksson s* umea

S Eriksson Umea plant sci ctr

S Eriksson Geriatr Ctr

Staffan Eriksson at the same department published as S Eriksson

Variants of a department Dept Community Med & Rehabil Inst Community Med & Rehabil, Div Geriatr Dept Geriatr Med Geriatr ctr

Dept Community Med & Rehabil

Inst Community Med & Rehabil, Div Geriatr

Dept Geriatr Med

Geriatr ctr

Lost citations when using search limit author and address.

Author-address search solutions from Thomson ISI WoS Rantapaa$dahlq?ist s* $ equals one or zero characters * any number of characters ? is one single characters Author=astrom s Address=umea SAME dept nursing Retrieves at same line: umea univ dept nursing Doesn’t retrieve: Lund univ, Dept nursing Umea univ, Dept Ophthalmology

Rantapaa$dahlq?ist s*

$ equals one or zero characters

* any number of characters

? is one single characters

Author=astrom s Address=umea SAME dept nursing

Retrieves at same line: umea univ dept nursing

Doesn’t retrieve:

Lund univ, Dept nursing

Umea univ, Dept Ophthalmology

Lots of flaws when counting citations in Google Scholar. Many duplicates! Has no address field.

Why all these errors? Lacking (raw)data from the beginning. Unsuccessful indexing of (raw)data from the vendors Elsevier, Thomson och Google. Algorithms can’t solve lacking (raw)data, perhaps tune up some errors. It’s not a question about algo’s, it’s a question about structured well-indexed data.

Lacking (raw)data from the beginning.

Unsuccessful indexing of (raw)data from the vendors Elsevier, Thomson och Google.

Algorithms can’t solve lacking (raw)data, perhaps tune up some errors.

It’s not a question about algo’s, it’s a question about structured well-indexed data.

Multidisciplinary subject search

KeyWord Plus in ” Key Words Plus supplies additional search terms extracted from the titles of articles cited by authors in their bibliographies and footnotes”. ” ,,,provide a brief condensation of the major and minor themes Discussed”. Källa: www.garfield.library.upenn.edu/essays/v13p295y1990.pdf Records without references won't have KW+ - but more specifically, articles whose references are not linked to source items. In addition, it may be that those with very few linked references won't generate good candidates for KW+ either.

Hormone replacement therapy

Hormone replacement therapy in cited references of the article.

29% 37,7% 17% 16% 0,3%

Keywords/Thesaurus Subject words (and reference information) are collected from Elsevier databases och Medline: Geobase subject index Emtree - bara major headings are displyed! MeSH – bara major headings are displayed! FLX terms, WTA terms Regional Index Species Index Ei thesaurus Author keywords (uncontrolled)

Subject words (and reference information) are

collected from Elsevier databases och Medline:

Geobase subject index

Emtree - bara major headings are displyed!

MeSH – bara major headings are displayed!

FLX terms, WTA terms

Regional Index

Species Index

Ei thesaurus

Author keywords (uncontrolled)

Check Major MeSH Headings in a PubMed reference

Breast Neoplasms/surgery* is missing!

Emtree terms in Embase

Same reference in Scopus misses Emtree terms

Same thing with Compendex

… and same reference in Scopus

Lacking subject search No index search to get suggestions of terms. Thesauri are inconsistently integrated, sometimes no or just a selected part of (major) MeSH-terms, Emtree and Compendex terms. Uncontrolled terms mixed with controlled terms. No possibility to limit to controlled terms exclusive. No option for chosing thesauri. No mapping of terms, similar to PubMed and Embase. No possibility to explode a search for subterms. No integration of MeSH subheadings.

No index search to get suggestions of terms.

Thesauri are inconsistently integrated, sometimes no or just a selected part of (major) MeSH-terms, Emtree and Compendex terms.

Uncontrolled terms mixed with controlled terms. No possibility to limit to controlled terms exclusive.

No option for chosing thesauri.

No mapping of terms, similar to PubMed and Embase.

No possibility to explode a search for subterms.

No integration of MeSH subheadings.

Subject search in Google Scholar advanced search

Subject coverage in Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Google Scholar: An Empirical Study Portal: Libraries and the Academy Vol. 6, No. 2, pp. 127-141. From 47 databases 50 article titles were randomly collected from each database and compared with Google Scholar.

Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Google Scholar: An Empirical Study Portal: Libraries and the Academy Vol. 6, No. 2, pp. 127-141.

From 47 databases 50 article titles were randomly collected from each database and compared with Google Scholar.

Subject coverage Källa: Neuhaus, Chris et al (2006)

Database coverage Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 60%.

Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 60%.

Coverage evaluation of WoS and Scopus accomplished by Ylva Gavel, KIB, och Lars Iselid, Umeå UB. We’ve been matching ISSN against Ulrich’s journal database to exclude non-active titles, obsolete/invalid ISSN, titles not covered. It’s remarkably many. Study will be published in a scientific journal if accepted.

We’ve been matching ISSN against Ulrich’s journal database to exclude non-active titles, obsolete/invalid ISSN, titles not covered. It’s remarkably many.

Study will be published in a scientific journal if accepted.

Coverage evaluation of WoS and Scopus - How many active titles? Scopus 13.226 journals WoS 8.786 journals

Coverage evaluation of WoS and Scopus - How many active titles overlap and how many are unique? WoS Scopus 7.210 indexed both in Scopus and WoS All together 14.802 journals with unique ISSN.

Coverage evaluation of WoS and Scopus - How many active titles overlap? WoS covers 55% of Scopus Scopus covers 82% of WoS 49 % exist in both

Active titles in each database

Average overlap of WoS 51,8% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 55% 55% 46% 57% 36% 62%

Average overlap of Scopus 74,2% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 82% 91% 45% 71% 67% 89%

Coverage evaluation Scopus and Medline - How many active titles? Scopus 13.226 journals Medline 4.843 journals

Coverage evaluation Scopus and Medline - How many active titles is overlapping and how many unique? Medline Scopus 4.425 are indexed by both Scopus and Medline All together 13.644 journals with unique ISSN

Coverage evaluation Scopus and Medline - How many journals overlap? Medline covers 33% of Scopus Scopus covers 91% of Medline 32 % exist in both About one third of all unique journals in Scopus and Medline are indexed in both databases.

About one third of all unique journals in Scopus and Medline are indexed in both databases.

Coverage evaluation WoS vs. Medline - How many active titles? WoS 8.786 journals Medline 4.843 journals

Coverage evaluation WoS vs. Medline - How many active titles is overlapping and how many unique? Medline WoS 3.001 indexed By both WoS and Medline All together 10.682 journals with unique ISSN.

Coverage evaluation WoS vs. Medline - How many journals overlap? Medline covers 34% of WoS WoS covers 62% of Medline 28 % exist in both Nearly one third of all unique journals in WoS and Medline are indexed in both databases.

Nearly one third of all unique journals in WoS and Medline are indexed in both databases.

New options for author search: Scopus Author Identifier, WoS Author Finder Some improvements, some confusions!

Displays not just the address of the author retrieved, but also all co-authors addresses. Just displays institution, not department. Author Finder

Refine your results

What to do about the mess? Discuss the value of a citation and discuss alternative methods for measuring research. Discuss and measure the errors in the databases concerning citation search. By continuously evaluating citation databases we can put som pressure on the vendors to work more with improvements of the flaws.

Discuss the value of a citation and discuss alternative methods for measuring research.

Discuss and measure the errors in the databases concerning citation search.

By continuously evaluating citation databases we can put som pressure on the vendors to work more with improvements of the flaws.