Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

José Fernando F. Arruda, Luiz Nunes Oliveira, Marie-Anne Van Sluys - Evaluation of SciELO-Brazil journals by FAPESP: preliminary results

97 views

Published on

One of the basic functions of the SciELO Publishing Model is to follow up the performance of journals, national collections, the network and the overall program. In the context of national collections, which in most cases are financed by public resources and are highly selective regarding indexing, the good performance of journals is expected in line with the specific objectives of SciELO to contribute to their sustainable increase of editorial quality, visibility, use and impact. In addition to the specific objectives that apply to the entire network, national collections are governed by priorities determined by national policies and conditions.

The performance of journals and SciELO collections are evaluated by the following criteria:

• Institutionality, which refers to institutions responsible for the journals and their respective research communities as indicators of credibility and operational sustainability of journals;

• Good practices of editing and scholarly communication, which refers to the adherence to the SciELO indexing criteria that implies in adherence to the good practices of scholarly communication and adoption of innovations;

• Visibility, Use and Impact, which refer to the following contexts:
◦ Access and downloads indicators to articles’ full text files in HTML and PDF formats;
◦ Citations indicators or metrics considering different journal indexes;
◦ Web presence indicators or altmetrics.

The scope proposed for this working group encompasses the analysis of journals performance in accordance with the above criteria, taking into account the specificities of different thematic areas and different countries. The analysis and discussion of these three dimensions will be conducted by scholarly communication and bibliometrics experts with the support of representatives of national collections, journal editors and specialists.

Published in: Science
  • Be the first to comment

  • Be the first to like this

José Fernando F. Arruda, Luiz Nunes Oliveira, Marie-Anne Van Sluys - Evaluation of SciELO-Brazil journals by FAPESP: preliminary results

  1. 1. Evaluation of SciELO-Brazil journals by FAPESP: preliminary results J. Roberto F. Arruda1 Luiz Nunes Oliveira2 Marie-Anne Van Sluys3 1 Professor, FEM-Unicamp 2 Professor, IFSC-USP 3 Professor, IB-USP Special Advisors of the Scientific Director The São Paulo Research Foundation - FAPESP 115/10/2018 fapesp-short-pres-20150916.pptx; © C.H. Brito Cruz e Fapesp
  2. 2. State of São Paulo, Brazil 2 44 Million people 32% of Brazil’s GDP (500bn USD) >40% of Brazilian science 13% of State budget to HE and R&D 1.68% GDP for R&D 15/10/2018 © C.H. Brito Cruz e Fapesp
  3. 3. © C.H. Brito Cruz e Fapesp 315/10/2018 Science, Technology and Innovation in São Paulo
  4. 4. © C.H. Brito Cruz e Fapesp 415/10/2018 Science, Technology and Innovation in São Paulo
  5. 5. © C.H. Brito Cruz e Fapesp 515/10/2018 Science, Technology and Innovation in São Paulo
  6. 6. © C.H. Brito Cruz e Fapesp 615/10/2018 Science, Technology and Innovation in São Paulo
  7. 7. © C.H. Brito Cruz e Fapesp 715/10/2018 Science, Technology and Innovation in São Paulo
  8. 8. © C.H. Brito Cruz e Fapesp 815/10/2018 Fapesp: São Paulo Research Foundation
  9. 9. Fapesp: São Paulo Research Foundation • Created in 1962 by the State of São Paulo • 1% of all state revenues • Mission: “Support research in all fields of knowledge” 15/10/2018 9© C.H. Brito Cruz e Fapesp *PPP: Purchasing Power Parity
  10. 10. Fapesp: São Paulo Research Foundation 15/10/2018 10© C.H. Brito Cruz e Fapesp
  11. 11. Fapesp: São Paulo Research Foundation 15/10/2018 11© C.H. Brito Cruz e Fapesp
  12. 12. © C.H. Brito Cruz e Fapesp 1215/10/2018 Fapesp: São Paulo Research Foundation Internships abroad of scholars
  13. 13. © C.H. Brito Cruz e Fapesp 1315/10/2018 International agreements Fapesp: São Paulo Research Foundation
  14. 14. Fapesp 1410/15/2018 Fapesp: São Paulo Research Foundation International agreements
  15. 15. Fapesp: São Paulo Research Foundation 15/10/2018 15© C.H. Brito Cruz e Fapesp
  16. 16. SciELO funding by Fapesp 15/10/2018 16© C.H. Brito Cruz e Fapesp 0.00 5,000,000.00 10,000,000.00 15,000,000.00 20,000,000.00 25,000,000.00 30,000,000.00 2003-2005 2006-2008 2008-2011 2011-2013 2013-2016 2016-2019 R$ Abel Laerte PackerRogerio Meneghini Need to evaluate outcome
  17. 17. Fapesp proposal review scheme 15/10/2018 17© C.H. Brito Cruz e Fapesp
  18. 18. Fapesp proposal review scheme 15/10/2018 18© C.H. Brito Cruz e Fapesp Area panels (116) Agricultural and Veterinary Sciences (9) Architecture and Urbanism (2) Astronomy (1) Biology (14) Chemistry (3) Computer Science (3) Economics, Business and Public Administration (3) Engineering (12) Geosciences (4) Health Sciences (23) Humanities and Social Sciences (16) Mathematics (3) Physics (5) Research for Innovation (8) Adjunct panels (23) Humanities and Social Sciences, Architecture and Economy (3) Life Sciences (7) Physics, Mathematics, Chemistry and Engineering (4) Research for Innovation (5) Special Programs and Collaboration in Research (3) Multiuser Equipments (1)
  19. 19. Fapesp evaluation of SciELO journals 15/10/2018 19© C.H. Brito Cruz e Fapesp FAPESP & SciELO joint discussion on EDP • Defining key questions to the Editor • Significance & Sustainability • Best practices • Journal metrics evolution • Future Goals up to 2021 • Instructions for Editorial Development Plan (EDP)
  20. 20. FAPESP & SciELO: joint discussion 15/10/2018 20© C.H. Brito Cruz e Fapesp FAPESP & SciELO joint discussion on evaluation form • A lot – Sufficiently – Reasonably – Little - Nothing • Criterion 1. Editorial Board • Criterion 2. Area/Field Significance • Criterion 3. International Visibility • Criterion 4. Institutional & Sustainability • General Recommendation
  21. 21. Fapesp evaluation of SciELO journals 15/10/2018 21© C.H. Brito Cruz e Fapesp • Message sent to the Editors (11 June 2018) • EDP sent by July 10 postponed to July 15 • July-August evaluation by the area panels
  22. 22. Fapesp evaluation of SciELO journals 15/10/2018 22© C.H. Brito Cruz e Fapesp Panel PDEs Evaluated Journals EVALUATION Total VG G R L N N/A Agri/Vet 28 16 3 6 6 1 0 0 16 Bio 15 10 4 4 2 0 0 0 10 Health 71 18 7 6 7 3 1 0 24 Natural Sci 8 8 4 2 1 1 0 0 8 Eng 10 10 5 3 2 0 0 0 10 Human Sci 61 28 19 7 0 0 0 2 28 Social 30 7 3 2 1 0 0 1 7 Lang/Arts 10 0 0 0 0 0 0 0 0 Multi 1 0 0 0 0 0 0 0 0 OTAL 234 97 45 30 19 5 1 3 103 0 5 10 15 20 25 30 35 40 45 50 VG G R L N N/A General figures 35% 77% 6%
  23. 23. Fapesp: Evaluation of SciELO journals 15/10/2018 23© C.H. Brito Cruz e Fapesp • Q1a: Editor(s) are internationally recognized in the field of the journal? • Q2a: Journal reputation (quality of the papers and review) • Q2b: Would you recommend publishing in this journal? • Q3c: Impact indicators are good compared to other journals in this field? • Q4h: Final recommendation about keeping the journal in SciELO
  24. 24. Evaluation: Agriculture and veterinary 15/10/2018 24© C.H. Brito Cruz e Fapesp Q1a Q2a Q2b Q3c Q1a Editor Q2a Journal Q2b Would you publish? Q3c IF
  25. 25. Evaluation: Agriculture and veterinary 15/10/2018 25© C.H. Brito Cruz e Fapesp Q4h SJR Q Q4h Final recommendation
  26. 26. Evaluation: Natural Sciences and Engineering 15/10/2018 26© C.H. Brito Cruz e Fapesp Q1a Q2a Q2b Q3c Q1a Editor Q2a Journal Q2b Would you publish? Q3c IF
  27. 27. Evaluation: Natural Sciences and Engineering 15/10/2018 27© C.H. Brito Cruz e Fapesp Q4h SJR Q Q4h Final recommendation
  28. 28. Evaluation: Life Sciences 15/10/2018 28© C.H. Brito Cruz e Fapesp Q1a Q2a Q2b Q3c Q1a Editor Q2a Journal Q2b Would you publish? Q3c IF
  29. 29. Evaluation: Life Sciences 15/10/2018 29© C.H. Brito Cruz e Fapesp Q4h SJRQ Q4h Final recommendation
  30. 30. Evaluation: Social and Human Sciences 15/10/2018 30© C.H. Brito Cruz e Fapesp Q1a Q2a Q2b Q3c Q1a Editor Q2a Journal Q2b Would you publish? Q3c IF
  31. 31. Evaluation: Social and Human Sciences 15/10/2018 31© C.H. Brito Cruz e Fapesp Q4h SJR Q Q4h Final recommendation
  32. 32. Correlations: cits/doc SciELO x Q4h
  33. 33. Correlations: cits/doc WoS x Q4h
  34. 34. Correlations: Q1a (Editor) x Q4h
  35. 35. Correlations: Q2a (journal) x Q4h
  36. 36. Correlations: Q2b (w y publish?) x Q4h
  37. 37. Correlations: SciELO/WoS x Q4h
  38. 38. 200 2005 2010 2015 Examples: Revista Bras. Ciências Sociais
  39. 39. 500 2005 2010 2015 Examples: Rodriguésia
  40. 40. 500 2005 2010 2015 Examples: Scientia Agricola
  41. 41. 120 2011 2015 Examples: LA J. of Solids and Structures
  42. 42. Fapesp evaluation of SciELO journals 15/10/2018 42© C.H. Brito Cruz e Fapesp • Very preliminary analysis and conclusions • Difficult to obtain unbiased evaluation, as experts are almost always involved with the journals (past or present) • Uneven quality of evaluations • Interesting suggestions by many reviewers will be helpful to journals • What to do with journals that have not send PDE?
  43. 43. Fapesp evaluation of SciELO journals 15/10/2018 43© C.H. Brito Cruz e Fapesp Contact information: José Roberto F. Arruda <JRArruda@fapesp.br> Luiz Nunes de Oliveira <LNunes@fapesp.br> Marie-Anne Van Sluys <MSluys@fapesp.br>
  44. 44. Fapesp evaluation of SciELO journals 15/10/2018 44© C.H. Brito Cruz e Fapesp • Instructions for EDP preparation (up to 5 pages) – Editorial practices – Institutionality – Sustainability – Singularity (relevance) – Visibility and impact – Technology – Editorial goals • Evaluation form (4 criteria) – Criterion 1: Quality of the Editorial Board and reviewers – Criterion 2: Reputation of the journal – Criterion 3: International visibility and impact factors – Criterion 4: Institutionality and sustainability

×