Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The SSH conundrum: A matter of audiences

227 views

Published on

Presentation given in Paris, France on May 23 within a workshop organized by HCÉRES-OST.

Published in: Education
  • Be the first to comment

  • Be the first to like this

The SSH conundrum: A matter of audiences

  1. 1. The SSH conundrum A matter of audiences? Nicolas Robinson-Garcia
  2. 2. 1. Context 2. Framing the problem 3. Current applications 4. Evidences of misadjustment 5. Considerations Agenda
  3. 3. SSH -> the source of bibliometricians and research managers’ headaches: •Tends to underperform in comparison with other fields •Less productive, less impact •Perception on their purpose and usefulness is influenced by limitations in its assessment Context
  4. 4. These limitations are useful for some types of arguments: 1. Limiting funding 2. Hiding mediocrity Context
  5. 5. •No international core of SSH literature •Multiple outputs •Multiple purposes •Lack of normalization in quality control •Differing citation patterns •SSH as a “basket concept” Listing shortcomings
  6. 6. Quick revision 1. The four literatures of the Social Sciences (Hicks, 2005) 2. Audiences (Nederhof, 2006) 3. The rural vs. the urban (van Leeuwen et al., 2017 adapted from Becher & Trowler, 2002) Framing the problem
  7. 7. The four literatures Hicks, 2005 International journals Books National literature Non-scholarly literature
  8. 8. The four literatures Hicks, 2005 International journals Books National literature Non-scholarly literature
  9. 9. The four literatures Hicks, 2005 •The argument is focused on scientists’ outputs •Some overlap is acknowledged but it is presented as problem of coverages •It does not question the use of citations, at least not explicitly
  10. 10. Audiences Nederhof, 2001 International journals Books National literature Non-scholarly literature
  11. 11. Audiences Nederhof, 2001 Three audiences 1. The global scientific communities 2. Local experts 3. The general public
  12. 12. The rural vs. the urban van Leeuwen et al. 2017 SSH vs. STEM 1. Publication culture Differing outputs and references 2. Language of publication Differing targeted outreach 3. Locality of topics Differing targeted impact
  13. 13. Tweaks to correct misadjustments •Addition of books and chapter to bibliometric databases (e.g., Book Citation Index) •Addition of new data sources (e.g., Google Books, national databases) •Addition of indicators (e.g., book reviews, library holdings, altmetrics) •Rankings (e.g., publishers’ prestige, citation impact) Confronting the problem
  14. 14. Some examples from Spain •Journals – Criteria based on inclusion in international databases •Citations remain as main proxy for impact (informally others can be alleged) •Publisher rankings – Elea is the expert! •Other impact proxies for books – Book reviews, translations Confronting the problem
  15. 15. CASE I Department A – Contemporary History • Data source: Dialnet • 93 Faculty • 2,724 documents Data kindly provided by Carlos B. Amat, derived from Cañibano et al. 2018 But still…
  16. 16. Outputs But still… Articles Chapters Books
  17. 17. Outreach (only journals) International Domestic Regional Institutional Departmental But still…
  18. 18. Outreach (only journals) But still… Publications in Scopus in WoS Prof. A 288 2 9 Prof. B 207 2 14 Prof. C 124 0 0 Prof. D 100 4 0
  19. 19. CASE II Monographs published by University B • Data source: CRIS + PlumX + Web of Science • Period 2010-2016 • 2,957 books Torres-Salinas, Robinson-Garcia & Gorraiz, 2017 But still…
  20. 20. Impacts + Outputs But still…
  21. 21. There are too many aspects to consider to be able to address all of them with a single approach •Audiences (geographical outreach, stakeholders) •Outputs (books, journal articles, social media, press media) •Impacts (educational, scientific, social) But still…
  22. 22. Considerations In a convoluted world, nothing lasts forever: “library holdings seem to be the most promising proxy of scholarly impact” Torres-Salinas et al. 2018 For how long? Will books continue being relevant? Will they migrate to electronic format?
  23. 23. • Productivity scales as a means to determine what is good – Benchmarking • Greater capability on developing automatic methods – Connecting APIs from different sources • Great opportunities for developing national data sources and unique instruments • More creativity is necessary • Sometimes less data is better than more data • Micro analyses are context-driven • Impact can be stakeholder-driven (analyze communities) • Contextual factors may lead to identify spillovers Considerations
  24. 24. One suggestion Considerations “The introduction of knowledge about the process into assessment procedures will also help us to understand how (potential) social impact is being achieved.” Spaapen & Drooge, 2011 Researchers’ context as a proxy of engagement potential Social networks of scientists as proxy for social outreach
  25. 25. Global Local Geographical distance One suggestion Considerations
  26. 26. One suggestion Considerations Public sector NGO Unknown Academia Private sector Politicians Institutional affiliation
  27. 27. •Evaluation vs. Monitoring • Evaluation design must be driven by purpose and context (better and more varied approaches and methods) • Monitoring requires control and exhaustiveness (better and more comprehensive sources) •Combinations of creativity and empiricism • Point scales must not be aleatory but evidence-based • Translating similar methods from one field to another is not enough The end of universal approaches
  28. 28. Thank you! Web: http://nrobinsongarcia.com Email: elrobinster@gmail.com Twitter: @nrobinsongarcia

×