One Entry to Research:   critical assessment of Web of Science(WoS), Scopus and Google Scholar(GS) 10.20-11.00 Thursday 31...
BIBSAM-project from 1/1 2006 – 31/8 2006: One Entry to Research – critical assessment of Web of Science, Scopus och Google...
Evaluations of multidisciplinary citation databases <ul><li>Citation search </li></ul><ul><ul><li>Author search </li></ul>...
Questions for the libraries? <ul><li>Shall we keep Web of Science or supplement with or change to Scopus?  </li></ul><ul><...
Answers from a librarian! <ul><li>No, we should not use Google Scholar exclusively, which has lots of flaws for both citat...
Rantapaa* 8 name variants
Rantapaa S* 2 additional name variants Of course much of the incorrect author spellings in Cited Ref Search could be blame...
The Lancet-article? Rantapaa Dahlqvist or Rantapaa-Dahlqvist
The Lancet-article? Rantapaadahlqvist
The Lancet-article?  Rantapaa-Dahlquist
The Lancet article! Original e-journal article
Lancet article in
Rantapaa* S
Lancet article in Dahlqvist, S.R.
Lancet article in
Due to all incorrect information in author and address fields  I recommend to compare with a publication list from the res...
Author search for finding times cited <ul><li>AG Fisher’s affiliation is Umeå but has been in Colorado before. </li></ul><...
Has AG Fisher published articles as A Fisher? 8 hits but no one by Anne G Fisher
Two articles with spelling A Fisher with search limits  Am J Occup Ther 1987  (where address is not Umea or Colorado)
Two AG Fisher has published in same journal. But none with address Colorado or Umea. Right Anne Fisher! False Anne Fisher!
<ul><li>Is it of great value to compare with publication lists?  </li></ul><ul><li>Yes, too often. </li></ul><ul><li>Do re...
One of the articles by PO Sandman does not exist in his publication list. It’s cited 2 times in WoS.
These articles from WoS are not existing in the publication list of S Eriksson. 14 times cited.
Bad implementation of data! Address is missing for Rantapää
… but in the original e-journal article, it’s there!
Search with limit umea could produce false amount of citations!! S Astrom umea
S Astrom Dept ophthalmol
Eriksson s* umea
S Eriksson Umea plant sci ctr
S Eriksson Geriatr Ctr
Staffan Eriksson at the same department published as S Eriksson
Variants of a department <ul><li>Dept Community Med & Rehabil </li></ul><ul><li>Inst Community Med & Rehabil, Div Geriatr ...
Lost citations when using search limit author and address.
Author-address search solutions from Thomson ISI WoS <ul><li>Rantapaa$dahlq?ist s* </li></ul><ul><ul><li>$ equals one or z...
Lots of flaws when counting citations in Google Scholar. Many duplicates! Has no address field.
Why all these errors? <ul><li>Lacking (raw)data from the beginning. </li></ul><ul><li>Unsuccessful indexing of (raw)data f...
Multidisciplinary subject search
KeyWord Plus in  ” Key Words Plus  supplies additional search terms extracted from the titles of articles cited by authors...
Hormone replacement therapy
Hormone replacement therapy  in cited references of the article.
29% 37,7% 17% 16% 0,3%
Keywords/Thesaurus <ul><li>Subject words (and reference information) are  </li></ul><ul><li>collected from Elsevier databa...
Check Major MeSH Headings in a PubMed reference
Breast Neoplasms/surgery* is missing!
Emtree terms in Embase
Same reference in Scopus misses Emtree terms
Same thing with Compendex
… and same reference in Scopus
Lacking subject search  <ul><li>No index search to get suggestions of terms.  </li></ul><ul><li>Thesauri are inconsistentl...
Subject search in Google Scholar advanced search
Subject coverage in  <ul><li>Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Goo...
Subject coverage Källa: Neuhaus, Chris et al (2006)
Database coverage <ul><li>Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 6...
Coverage evaluation of WoS and Scopus accomplished by Ylva Gavel, KIB, och Lars Iselid, Umeå UB. <ul><li>We’ve been matchi...
Coverage evaluation of WoS and Scopus  - How many active titles?   Scopus  13.226 journals WoS 8.786 journals
Coverage evaluation of WoS and Scopus  -  How many active titles overlap and how many are unique?   WoS Scopus 7.210  inde...
Coverage evaluation of WoS and Scopus  -  How many active titles overlap? WoS covers 55% of Scopus Scopus covers 82% of Wo...
Active titles in each database
Average overlap of WoS 51,8% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 55% 55% 46% 57% 36% 62%
Average overlap of Scopus 74,2% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 82% 91% 45% 71% 67% 89%
Coverage evaluation Scopus and Medline -  How many active titles? Scopus  13.226 journals Medline 4.843 journals
Coverage evaluation Scopus and Medline  - How many active titles is overlapping and how many unique?   Medline Scopus 4.42...
Coverage evaluation Scopus and Medline  -  How many journals overlap? Medline covers 33% of Scopus  Scopus covers 91% of M...
Coverage evaluation WoS vs. Medline -  How many active titles? WoS  8.786 journals Medline 4.843 journals
Coverage evaluation WoS vs. Medline  -  How many active titles is overlapping and how many unique?   Medline WoS 3.001  in...
Coverage evaluation WoS vs. Medline   -  How many journals overlap? Medline covers  34% of WoS WoS covers  62% of Medline ...
New options for author search: Scopus Author Identifier, WoS Author Finder Some improvements,  some confusions!
Displays not just the address of the author retrieved, but also all co-authors  addresses. Just displays institution, not ...
Refine your results
What to do about the mess? <ul><li>Discuss the value of a citation and discuss alternative methods for measuring research....
Upcoming SlideShare
Loading in...5
×

One Entry to Research: critical assessment of Web of Sceince, Scopus and Google Scholar.

4,430

Published on

Published in: Economy & Finance, Technology
2 Comments
9 Likes
Statistics
Notes
  • Excellent. You've shown your credibility on presentation with this slideshow. This one deserves thumbs up. I'm John, owner of www.freeringtones.ws/ . Perhaps I'll get to see more quality slides from you.

    Best wishes.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Outstanding display. Really clear together with helpful

    Janie
    http://financejedi.com
    http://healthjedi.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,430
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
2
Likes
9
Embeds 0
No embeds

No notes for slide

Transcript of "One Entry to Research: critical assessment of Web of Sceince, Scopus and Google Scholar."

  1. 1. One Entry to Research: critical assessment of Web of Science(WoS), Scopus and Google Scholar(GS) 10.20-11.00 Thursday 31 August BIBSAM workshop, National Library, Sthlm Speaker: Lars Iselid
  2. 2. BIBSAM-project from 1/1 2006 – 31/8 2006: One Entry to Research – critical assessment of Web of Science, Scopus och Google Scholar
  3. 3. Evaluations of multidisciplinary citation databases <ul><li>Citation search </li></ul><ul><ul><li>Author search </li></ul></ul><ul><ul><li>Address and department/institution search </li></ul></ul><ul><li>Subject search and coverage </li></ul><ul><li>Overlap and coverage between Scopus, WoS etc (Not GS) </li></ul>
  4. 4. Questions for the libraries? <ul><li>Shall we keep Web of Science or supplement with or change to Scopus? </li></ul><ul><li>Or do we need them at all when we have Google Scholar for free? </li></ul><ul><li>Is it sufficient to search for citations in Web of Science or should we also consider Scopus and Google Scholar? </li></ul><ul><li>Are these databases fullworthy as subject databases? </li></ul>
  5. 5. Answers from a librarian! <ul><li>No, we should not use Google Scholar exclusively, which has lots of flaws for both citation and subject search. </li></ul><ul><li>If we want to find more cited references concerning research from 1996 and present we have to consider Scopus (and Google Scholar even pre-1996) beside of Web of Science. </li></ul><ul><li>Google Scholar has often unique citations in comparision with Scopus and Web of Science. </li></ul>
  6. 6. Rantapaa* 8 name variants
  7. 7. Rantapaa S* 2 additional name variants Of course much of the incorrect author spellings in Cited Ref Search could be blamed on incorrect citation information from article authors.
  8. 8. The Lancet-article? Rantapaa Dahlqvist or Rantapaa-Dahlqvist
  9. 9. The Lancet-article? Rantapaadahlqvist
  10. 10. The Lancet-article? Rantapaa-Dahlquist
  11. 11. The Lancet article! Original e-journal article
  12. 12. Lancet article in
  13. 13. Rantapaa* S
  14. 14. Lancet article in Dahlqvist, S.R.
  15. 15. Lancet article in
  16. 16. Due to all incorrect information in author and address fields I recommend to compare with a publication list from the researcher. Publication lists
  17. 17. Author search for finding times cited <ul><li>AG Fisher’s affiliation is Umeå but has been in Colorado before. </li></ul><ul><li>5 AG Fisher with three first name initials </li></ul>
  18. 18. Has AG Fisher published articles as A Fisher? 8 hits but no one by Anne G Fisher
  19. 19. Two articles with spelling A Fisher with search limits Am J Occup Ther 1987 (where address is not Umea or Colorado)
  20. 20. Two AG Fisher has published in same journal. But none with address Colorado or Umea. Right Anne Fisher! False Anne Fisher!
  21. 21. <ul><li>Is it of great value to compare with publication lists? </li></ul><ul><li>Yes, too often. </li></ul><ul><li>Do researchers really manage to keep track of their articles? </li></ul><ul><li>No, many of them don’t. But we must tell them to do that if they want to hurry up their career. </li></ul>
  22. 22. One of the articles by PO Sandman does not exist in his publication list. It’s cited 2 times in WoS.
  23. 23. These articles from WoS are not existing in the publication list of S Eriksson. 14 times cited.
  24. 24. Bad implementation of data! Address is missing for Rantapää
  25. 25. … but in the original e-journal article, it’s there!
  26. 26. Search with limit umea could produce false amount of citations!! S Astrom umea
  27. 27. S Astrom Dept ophthalmol
  28. 28. Eriksson s* umea
  29. 29. S Eriksson Umea plant sci ctr
  30. 30. S Eriksson Geriatr Ctr
  31. 31. Staffan Eriksson at the same department published as S Eriksson
  32. 32. Variants of a department <ul><li>Dept Community Med & Rehabil </li></ul><ul><li>Inst Community Med & Rehabil, Div Geriatr </li></ul><ul><li>Dept Geriatr Med </li></ul><ul><li>Geriatr ctr </li></ul>
  33. 33. Lost citations when using search limit author and address.
  34. 34. Author-address search solutions from Thomson ISI WoS <ul><li>Rantapaa$dahlq?ist s* </li></ul><ul><ul><li>$ equals one or zero characters </li></ul></ul><ul><ul><li>* any number of characters </li></ul></ul><ul><ul><li>? is one single characters </li></ul></ul><ul><li>Author=astrom s Address=umea SAME dept nursing </li></ul><ul><ul><li>Retrieves at same line: umea univ dept nursing </li></ul></ul><ul><li>Doesn’t retrieve: </li></ul><ul><ul><li>Lund univ, Dept nursing </li></ul></ul><ul><ul><li>Umea univ, Dept Ophthalmology </li></ul></ul>
  35. 35. Lots of flaws when counting citations in Google Scholar. Many duplicates! Has no address field.
  36. 36. Why all these errors? <ul><li>Lacking (raw)data from the beginning. </li></ul><ul><li>Unsuccessful indexing of (raw)data from the vendors Elsevier, Thomson och Google. </li></ul><ul><li>Algorithms can’t solve lacking (raw)data, perhaps tune up some errors. </li></ul><ul><li>It’s not a question about algo’s, it’s a question about structured well-indexed data. </li></ul>
  37. 37. Multidisciplinary subject search
  38. 38. KeyWord Plus in ” Key Words Plus supplies additional search terms extracted from the titles of articles cited by authors in their bibliographies and footnotes”. ” ,,,provide a brief condensation of the major and minor themes Discussed”. Källa: www.garfield.library.upenn.edu/essays/v13p295y1990.pdf Records without references won't have KW+ - but more specifically, articles whose references are not linked to source items. In addition, it may be that those with very few linked references won't generate good candidates for KW+ either.
  39. 39. Hormone replacement therapy
  40. 40. Hormone replacement therapy in cited references of the article.
  41. 41. 29% 37,7% 17% 16% 0,3%
  42. 42. Keywords/Thesaurus <ul><li>Subject words (and reference information) are </li></ul><ul><li>collected from Elsevier databases och Medline: </li></ul><ul><ul><li>Geobase subject index </li></ul></ul><ul><ul><li>Emtree - bara major headings are displyed! </li></ul></ul><ul><ul><li>MeSH – bara major headings are displayed! </li></ul></ul><ul><ul><li>FLX terms, WTA terms </li></ul></ul><ul><ul><li>Regional Index </li></ul></ul><ul><ul><li>Species Index </li></ul></ul><ul><ul><li>Ei thesaurus </li></ul></ul><ul><ul><li>Author keywords (uncontrolled) </li></ul></ul>
  43. 43. Check Major MeSH Headings in a PubMed reference
  44. 44. Breast Neoplasms/surgery* is missing!
  45. 45. Emtree terms in Embase
  46. 46. Same reference in Scopus misses Emtree terms
  47. 47. Same thing with Compendex
  48. 48. … and same reference in Scopus
  49. 49. Lacking subject search <ul><li>No index search to get suggestions of terms. </li></ul><ul><li>Thesauri are inconsistently integrated, sometimes no or just a selected part of (major) MeSH-terms, Emtree and Compendex terms. </li></ul><ul><li>Uncontrolled terms mixed with controlled terms. No possibility to limit to controlled terms exclusive. </li></ul><ul><li>No option for chosing thesauri. </li></ul><ul><li>No mapping of terms, similar to PubMed and Embase. </li></ul><ul><li>No possibility to explode a search for subterms. </li></ul><ul><li>No integration of MeSH subheadings. </li></ul>
  50. 50. Subject search in Google Scholar advanced search
  51. 51. Subject coverage in <ul><li>Neuhaus, Chris, (2006) Ellen Neuhaus, Alan Asher and Clint Wrede The Depth and Breadth of Google Scholar: An Empirical Study Portal: Libraries and the Academy Vol. 6, No. 2, pp. 127-141. </li></ul><ul><li>From 47 databases 50 article titles were randomly collected from each database and compared with Google Scholar. </li></ul>
  52. 52. Subject coverage Källa: Neuhaus, Chris et al (2006)
  53. 53. Database coverage <ul><li>Each databse of 47 (with 2350 randomly selected articles) had a median and average coverage of 60%. </li></ul>
  54. 54. Coverage evaluation of WoS and Scopus accomplished by Ylva Gavel, KIB, och Lars Iselid, Umeå UB. <ul><li>We’ve been matching ISSN against Ulrich’s journal database to exclude non-active titles, obsolete/invalid ISSN, titles not covered. It’s remarkably many. </li></ul><ul><li>Study will be published in a scientific journal if accepted. </li></ul>
  55. 55. Coverage evaluation of WoS and Scopus - How many active titles? Scopus 13.226 journals WoS 8.786 journals
  56. 56. Coverage evaluation of WoS and Scopus - How many active titles overlap and how many are unique? WoS Scopus 7.210 indexed both in Scopus and WoS All together 14.802 journals with unique ISSN.
  57. 57. Coverage evaluation of WoS and Scopus - How many active titles overlap? WoS covers 55% of Scopus Scopus covers 82% of WoS 49 % exist in both
  58. 58. Active titles in each database
  59. 59. Average overlap of WoS 51,8% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 55% 55% 46% 57% 36% 62%
  60. 60. Average overlap of Scopus 74,2% Scopus WoS Medline Embase Compendex PsycINFO Sociological Abstracts 82% 91% 45% 71% 67% 89%
  61. 61. Coverage evaluation Scopus and Medline - How many active titles? Scopus 13.226 journals Medline 4.843 journals
  62. 62. Coverage evaluation Scopus and Medline - How many active titles is overlapping and how many unique? Medline Scopus 4.425 are indexed by both Scopus and Medline All together 13.644 journals with unique ISSN
  63. 63. Coverage evaluation Scopus and Medline - How many journals overlap? Medline covers 33% of Scopus Scopus covers 91% of Medline 32 % exist in both <ul><li>About one third of all unique journals in Scopus and Medline are indexed in both databases. </li></ul>
  64. 64. Coverage evaluation WoS vs. Medline - How many active titles? WoS 8.786 journals Medline 4.843 journals
  65. 65. Coverage evaluation WoS vs. Medline - How many active titles is overlapping and how many unique? Medline WoS 3.001 indexed By both WoS and Medline All together 10.682 journals with unique ISSN.
  66. 66. Coverage evaluation WoS vs. Medline - How many journals overlap? Medline covers 34% of WoS WoS covers 62% of Medline 28 % exist in both <ul><li>Nearly one third of all unique journals in WoS and Medline are indexed in both databases. </li></ul>
  67. 67. New options for author search: Scopus Author Identifier, WoS Author Finder Some improvements, some confusions!
  68. 68. Displays not just the address of the author retrieved, but also all co-authors addresses. Just displays institution, not department. Author Finder
  69. 69. Refine your results
  70. 70. What to do about the mess? <ul><li>Discuss the value of a citation and discuss alternative methods for measuring research. </li></ul><ul><li>Discuss and measure the errors in the databases concerning citation search. </li></ul><ul><li>By continuously evaluating citation databases we can put som pressure on the vendors to work more with improvements of the flaws. </li></ul>

×