Google Scholar as a research and evaluation tool
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Google Scholar as a research and evaluation tool

on

  • 878 views

Presentación realizada en la European Sumer for Scientometrics 2014. Viena, 10 de julio de 2014 http://www.scientometrics-school.eu/programme.html ...

Presentación realizada en la European Sumer for Scientometrics 2014. Viena, 10 de julio de 2014 http://www.scientometrics-school.eu/programme.html

Since its emergence in 2004, Google Scholar has attracted a huge interest in the scientific community. More recently, it has also drawn attention not only as a information source but also as a tool for evaluation purposes.
The launch of products such as Google Scholar Citations and Metrics or the recent agreement with Thomson Reuters' Web of Science shows that Google is already a major player in the scientific information market.
Its price (free), its huge coverage and the better attention to Social Science and Humanities (compared to commercial databases) has made Google Scholar a potentially valid source for bibliometrics in these areas. Nevertheless, Google Scholar (and their tools) presents many shortcomings that are necessary to know to perform reliable analysis.
In this session, we will briefly review Google Scholar pros and cons and will examine the usefulness of tools such as Citations and Metrics.

Statistics

Views

Total Views
878
Views on SlideShare
777
Embed Views
101

Actions

Likes
0
Downloads
7
Comments
0

3 Embeds 101

https://twitter.com 79
http://ec3metrics.com 21
https://tweetdeck.twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial LicenseCC Attribution-NonCommercial License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Google Scholar as a research and evaluation tool Presentation Transcript

  • 1. Google Scholar as a research and evaluation toolevaluation tool Álvaro Cabezas-Clavijo
  • 2. INDEX • Goals • GS as a research tool – Search engine – Alerts – Updates – Library + Cite • GS as a tool for evaluating research• GS as a tool for evaluating research – GS Citations (or Profiles) – GS Metrics – Bibliometric analysis • Data manipulation • Comparison to WOS/ Scopus • Conclusions
  • 3. To show GS possibilities to do research To show GS possibilities to evaluate research To point out strengths and weaknesses of GS as a source to perform bibliometric analysis GOALS To introduce Google Scholar as a valid source for carrying out research assessments (with caution!) in Social Sciences and Humanities (SSH)
  • 4. GS AS A TOOL FOR RESEARCHERSGS AS A TOOL FOR RESEARCHERS
  • 5. • GS is a widespread tool for doing research • Comprehensive source of academic literature • Widely known by scholars and students • Ease of use (just like google) GS AS A TOOL FOR RESEARCHERS • • Free and fast • Links to primary source (when it is freely available on the web or can be accessed from our institution)
  • 6. • Non-selective source (covers all material which meet the technical requirements) • No restrictions on language • No restrictions on document type • No discipline/areas bias GS AS A TOOL FOR RESEARCHERS (II) • No discipline/areas bias • Estimated size is 100 million documents100 million documents (Web of Science 50 million; Scopus 53 million)
  • 7. GS AS A TOOL FOR RESEARCHERS (III)
  • 8. • GS is not just an academic search engine but a “family” of tools for doing and evaluating research GS AS A TOOL FOR RESEARCHERS (IV) ALERTS UPDATES LIBRARY + CITE
  • 9. • Set up by user just clicking on the “Create alert” link when performing a search • Delivered by 2-4 days if there is fresh info to show • Authors or thematic searches can be configured (but not for journals) ALERTS journals) 1 2 3
  • 10. • Configure an alert for citations to particular papers or authors (in this case, if they have already set up a profile) ALERTS Following Christian’s new papers or citations
  • 11. • Alerts about papers of our interest automatically generated by Google • You only need to set up a GS profile. Algorithm takes into account: – Who we cite in our papers – Who cite us in their papers UPDATES – Who cite us in their papers – Titles of our papers – Co-authors – Content • A red bell indicates the number of new papers that might be of your interest
  • 12. UPDATESSome paper recommendations Ok, I have published about these topics, so I may be interested in these papers Ok, this paper cites me so it should be related to my research
  • 13. • Kind of reference manager • Save references in one click • Arrange references by using labels • Edit references & export them (but one by one!) • Can’t import references from other bibliographic software LIBRARY + CITE • Can’t import references from other bibliographic software •Cite easily from GS results: formats MLA, APA, Chicago •Get the formatted reference and copy/paste it into a Bibliography.
  • 14. • By default; 2 folders: My citations (actually My papers) and Cited by me • You can set up other LIBRARY + CITE • You can set up other labels/folders and save new references from GS searches
  • 15. GS as a tool for evaluating researchGS as a tool for evaluating research
  • 16. • GS provides different tools which can help in assessments of individuals, journals and more GS AS A TOOL FOR EVALUATORS (II) PROFILES METRICS SOURCE FOR BIBLIOMETRIC ANALYSES
  • 17. • Individual Web page with publication list and basic bibliometric indicators • Papers are ranked (by default) according to their number of citations • Also links to co-authors (if they have set up a profile) and PROFILES • Also links to co-authors (if they have set up a profile) and keywordskeywords Ranking of researchers by keywords/areas are generated through this classification TIP: there is always a field where you can be the most cited author! ☺
  • 18. • Easy to set up and edit • Strongly recommended for young researchers as it can increase their visibility • You will get an alert (maximum 3 times a week) every time you are cited PROFILES • 72,579 researchers with profile (March 2012) (Estimates by Ortega & Aguillo, 2013) • GS profiles-derived products – Some Spanish institutions are ranking their researchers according to GSC (Univ. Granada, CSIC,…) stimulus to set up a profile?
  • 19. COMMUNICATION: 886 RESEARCHERS
  • 20. SCIENTOMETRICS: 265 RESEARCHERS
  • 21. CSIC RANKING OF RESEARCHERS ACCORDING TO GSC
  • 22. • Easy to set up (just search for your papers) • Terrific tool for comparing researchers within a field or department • Automatically updated basic bibliometric data PROFILES • Data can be easily manipulated (a researcher can self-claim non-authored papers) • Can stimulate vanity and ego • Can generate no fair comparisons (for example,bibliometric data • No restrictions on source, language or area • Everyone can measure their performance (and their colleagues') comparisons (for example, researchers from different areas in a single univ) • Can generate unfair analysis by non-bibliometric experts STRENGTHS WEAKNESSES
  • 23. • GS service devoted to measure journal’s impact according to their h-index (est. April 2012) • Rankings are provided by language (nine languages) areas (8) and subcategories (313) (just for English-language journals) and journal title’s words (i.e, oncology) • Subject/area rankings are limited to top 20 journals and language METRICS • Subject/area rankings are limited to top 20 journals and language rankings are limited to top 100 journals. • Calculations take into account output for 5 years (currently 2008-2012) and citations received until july 2013. A new update is expected to be released in July 2014. • Journals with fewer than 100 papers published in 5 years or with no citation (h-index=0) are excluded • arXiv and other repositories’ collections are also included
  • 24. • Free product to compare and rank journals • We can get impact information about non-JCR journals and about national and SSH publications • Transparency: citations for every paper that contributes to the h-index can be checked • Methodological inconsistencies such as comparing journals from different areas • Lack of proper bibliographic control (duplicates, “dirty” data,…) • No selection criteria • No action against data manipulation METRICS can be checked • High correlation with JCR Impact factor (0.82) • Simple to use and understand by any scholar • Can be easily replicated • No action against data manipulation • No info about new updates (expected to be updated yearly) • Just top results are presented • H index favours journals with high production STRENGTHS WEAKNESSES
  • 25. • GS is estimated to contain about 100 million documents (87% of scholarly papers on the web) Khabsa & Giles (2014) • So, it is the largest source for bibliometric analysis that we can use SOURCE FOR BIBLIOMETRIC ANALISYS can use • GS-based products can also be used to help with research assessment – Publish or Perish (software for bibliometric analysis) – H-Index Scholar (ranking of spanish researchers) – Google Chrome add-ons – …
  • 26. PUBLISH OR PERISH
  • 27. H-INDEX SCHOLAR
  • 28. GOOGLE CHROME ADD-ONS
  • 29. • Many things to sort out (unknown coverage, duplicates, “dirty” data) but GS can be useful for: Non-covered areas by WoS… Non-English research… SOURCE FOR BIBLIOMETRIC ANALISYS Non-English research… …Which means basically Social Sciences and Humanities institutions and individuals which lack access to WoS/Scopus
  • 30. GS data can be easily manipulated • Just need to upload an “academic” paper to a institutional repository, university webpage or similar • “Academic” means a paper with title, authors, abstract and of course, bibliographic references • GS will count the citations from these fake papers, modifying• GS will count the citations from these fake papers, modifying indicators of all cited scholars and journals. Delgado, Robinson & Torres (2014)
  • 31. • Small diferences between rankings generated by GS, WoS and Scopus • Intensive Care Medicine Spearman correlation of 0.99 for journals and 0.93 for top researchers Comparison to Wos / Scopus JOURNALSJOURNALS Cabezas & Delgado (2013) • Spearman correlation for Communication Journals: 0.895 (GSM-WoS); 0.879 (GSM-Scopus) • Spearman correlation for Economics & Business Journals: 0.718 (GS-WoS) JOURNALS Delgado & Repiso (2013) Harzing & der Wal (2009)
  • 32. • Easy to use, fast and free • Largest source for assessment purposes • Provides information about areas and materials not covered by • Lack of bibliographic control • Data can be manipulated • No possibilities to massive info download • Time-consuming, data needs to SOURCE FOR BIBLIOMETRIC ANALISYS and materials not covered by traditional bibliometric products • Rankings are very similar to those offered by WoS / Scopus • Possibility to build new products based on GS data • Time-consuming, data needs to be “cleaned” • GS and their services could be closed down in any moment STRENGTHS WEAKNESSES
  • 33. ConclusionsConclusions
  • 34. • Bibliometricians need to find other products to complement WoS / Scopus-based evaluations (GS, ERIH, national products,…) • What if subscriptions to traditional databases are cancelled? – Spain paying 4-6 million Euros to Thomson Reuters yearly for national access to WoS from public research organizations CONCLUSIONS – Spain paying 4-6 million Euros to Thomson Reuters yearly for national access to WoS from public research organizations • SSH deserve to be measured (prominent rankings such as Shanghai or Leiden ignore Humanities and a huge part of Social Sciences!) • GS tools are far from perfect but provide us with some valuable information
  • 35. • Bibliometricians take usually the “easy way”; only measure what can be easily measured with databases such as WoS or Scopus • A challenge for bibliometricians is to get the most out of the existing tools to evaluate SSH (and to develop new ones!) • GS can be used as a complementary or alternative source to CONCLUSIONS • GS can be used as a complementary or alternative source to shed some light on SSH • Use with extreme caution, especially when assessing individuals • Keep an eye on the future: Institutions might be demanding GS-based analysis soon.
  • 36. Further readingsFurther readings
  • 37. • Delgado-López-Cózar, E., & Cabezas-Clavijo, Á. (2013). Ranking journals: could Google Scholar Metrics be an alternative to Journal Citation Reports and Scimago Journal Rank?. Learned publishing, 26(2), 101-113. • Delgado López-Cózar, E., Robinson-García, N., & Torres-Salinas, D. (2014). The Google Scholar Experiment: how to index false papers and manipulate bibliometric indicators. Journal of the Association for Information Science and Technology, 65(3), 446-454. • Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An FURTHER READINGS • Harzing, A. W., & Van Der Wal, R. (2009). A Google Scholar h-index for journals: An alternative metric to measure journal impact in economics and business. Journal of the American Society for Information Science and Technology, 60(1), 41-46. • Khabsa, M., & Giles, C. L. (2014). The Number of Scholarly Documents on the Public Web. PloS one, 9(5), e93949. • Ortega, J. L., & Aguillo, I. F. (2013). Institutional and country collaboration in an online service of scientific profiles: Google Scholar Citations. Journal of Informetrics, 7(2), 394-403.
  • 38. • BLOG: Google Scholar Digest: analysis of papers dealing with GS http://googlescholardigest.blogspot.com.es/ • TWITTER: Follow the #googlescholar hashtag • SOFTWARE: Publish or Perish http://www.harzing.com/pop.htm • PRODUCTS: FURTHER MATERIALS • PRODUCTS: • GOOGLE SCHOLAR CITATIONS http://scholar.google.com/citations • GOOGLE SCHOLAR METRICS http://scholar.google.com/citations?view_op=top_venues&hl=en
  • 39. Google Scholar as a research and evaluation toolevaluation tool Álvaro Cabezas-Clavijo Thank you!