Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using Google Scholar in research 
evaluation of social science 
programs 
Ad Prins 
Rodrigo Costas 
Thed van Leeuwen 
Paul...
Bibliometrics for Social Science 
Programs? 
• Coverage in WoS hampers bibliometric 
analyses 
• Deans of social science f...
Selection of Data 
• Ed/Ped: 13 programs of six universities, 2006- 
2011 
• Anthropology: 5 programs of 5 universities, 
...
Google Scholar Data Retrieval and 
Validity test 
1st order 
data 
2nd 
order 
data 
Using Scholar in Research Evaluation ...
Data quality 
Total number of 
verifiable citations 
Percentage of 
total found 
citations 
Ed/Ped 22887 89,8% 
Anthropolo...
Citing sources in Scholar 
Using Scholar in Research Evaluation 
September 3, 2014 6 
STI/ENID 2014, Leiden 
Source Anthro...
Cited documents 
Using Scholar in Research Evaluation 
September 3, 2014 7 
STI/ENID 2014, Leiden
Representation in WoS 
Using Scholar in Research Evaluation 
September 3, 2014 8 
STI/ENID 2014, Leiden
Comparing GS and WoS (1) 
Citations in GS Citations in GS, 
Academic 
sources 
Using Scholar in Research Evaluation 
Septe...
Comparing GS and WoS (2) 
Using Scholar in Research Evaluation 
September 3, 2014 10 
STI/ENID 2014, Leiden 
Anthropology:...
Comparing GS and WoS (3) 
Using Scholar in Research Evaluation 
September 3, 2014 11 
STI/ENID 2014, Leiden 
Ed/Ped: 
Most...
Issues in using GS 
• Workload and important data limitations 
• Possibilities for field normalization 
• What is a citati...
Conclusions 
• Google Scholar is a valuable resource: 
– The amount of information in Google Scholar is much 
larger than ...
Ad Prins 
Support in ResearchManagement 
www.adprins.nl 
September 3, 2014 
Using Scholar in Research Evaluation 
STI/ENID...
Upcoming SlideShare
Loading in …5
×

Using google scholar in research evaluation of social science research

591 views

Published on

Presentation for the STI/ENID conference 2014, september, Leiden, The Netherlands

Published in: Science
  • Be the first to comment

Using google scholar in research evaluation of social science research

  1. 1. Using Google Scholar in research evaluation of social science programs Ad Prins Rodrigo Costas Thed van Leeuwen Paul Wouters
  2. 2. Bibliometrics for Social Science Programs? • Coverage in WoS hampers bibliometric analyses • Deans of social science faculties in the Netherlands: Experiment for a bibliometric study in two research assessments in SSH • Google Scholar (GS) based for two fields – Education and Pedagogical Sciences (Ed/Ped) – Anthropology Using Scholar in Research Evaluation September 3, 2014 2 STI/ENID 2014, Leiden
  3. 3. Selection of Data • Ed/Ped: 13 programs of six universities, 2006- 2011 • Anthropology: 5 programs of 5 universities, 2004-2012 • Selection of publications – Ed/Ped: directors send in 10 publications – Anthr.: 5-10 most cited publications depending on size of program Using Scholar in Research Evaluation September 3, 2014 3 STI/ENID 2014, Leiden
  4. 4. Google Scholar Data Retrieval and Validity test 1st order data 2nd order data Using Scholar in Research Evaluation September 3, 2014 4 STI/ENID 2014, Leiden
  5. 5. Data quality Total number of verifiable citations Percentage of total found citations Ed/Ped 22887 89,8% Anthropology 8092 89,7% Verifiable (2nd order) citation data in GS: Proper internet address (URL) to journal, academic publisher, university library, acknowledged repository (Mendeley, SSRN, Academia.edu etc), academic societies etc. Using Scholar in Research Evaluation September 3, 2014 5 STI/ENID 2014, Leiden
  6. 6. Citing sources in Scholar Using Scholar in Research Evaluation September 3, 2014 6 STI/ENID 2014, Leiden Source Anthro-pology # Anthro-pology % Ed/Ped # Ed/Ped % Academic sources 4573 56.5% 14470 63.2% University Libraries 1677 20.7% 4573 20.0% Repositories other than ULs 616 7.6% 1236 5.4% Other Sources 327 4.0% 1085 4.7% Unknown 899 11.1% 1523 6.7% Total 8092 100.0% 22887 100.0% 56% to 63% of total volume of citations is found in academic sources (scientific journals and academic publishers) About 20% of the citations are located in University Libraries
  7. 7. Cited documents Using Scholar in Research Evaluation September 3, 2014 7 STI/ENID 2014, Leiden
  8. 8. Representation in WoS Using Scholar in Research Evaluation September 3, 2014 8 STI/ENID 2014, Leiden
  9. 9. Comparing GS and WoS (1) Citations in GS Citations in GS, Academic sources Using Scholar in Research Evaluation September 3, 2014 9 STI/ENID 2014, Leiden Citations in WoS Anthropology 8092 4573 1097 Education & 22887 14470 8870 Pedagogical Sciences • Information in GS exceeds that of WoS even when limited to academic sources (scientific journals and academic publishers)
  10. 10. Comparing GS and WoS (2) Using Scholar in Research Evaluation September 3, 2014 10 STI/ENID 2014, Leiden Anthropology: programs differ in benefitting from using GS
  11. 11. Comparing GS and WoS (3) Using Scholar in Research Evaluation September 3, 2014 11 STI/ENID 2014, Leiden Ed/Ped: Most programs have similar results, but some don’t
  12. 12. Issues in using GS • Workload and important data limitations • Possibilities for field normalization • What is a citation? Shifting definition. • Choosing a proper selection base • The place of bibliometry in research evaluations Using Scholar in Research Evaluation September 3, 2014 12 STI/ENID 2014, Leiden
  13. 13. Conclusions • Google Scholar is a valuable resource: – The amount of information in Google Scholar is much larger than in WoS: 2,5 to 8 times – Social science programs may benefit • But in using Google Scholar… – Data reliability control and access are an important issue – Only small/medium-scale studies are currently possible – New forms for field normalization are necessary – How much is Google Scholar indexing from the total output? Using Scholar in Research Evaluation September 3, 2014 13 STI/ENID 2014, Leiden
  14. 14. Ad Prins Support in ResearchManagement www.adprins.nl September 3, 2014 Using Scholar in Research Evaluation STI/ENID 2014, Leiden 14 Rodrigo Costas Thed van Leeuwen Paul Wouters Centre for Science and Technology Studies, Leiden www.cwts.nl

×