Berlin 6 Open Access Conference: Julianne Nyhan

  • 651 views
Uploaded on

www.berlin6.org

www.berlin6.org

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
651
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Evaluation, Quality and Impact of Open Access Publications
    • Chair: Julianne Nyhan (European Science Foundation)
    • Susan Murray (Director, African Journals online)
    • Christiane Fritze (Berlin-Brandenburg Academy of Sciences and Humanities)
  • 2.
    • European Reference Index
    • for the Humanities – ERIH
    • Better access to and assessment of Humanities research in Europe and beyond
    • Dr Julianne Nyhan
    • [email_address]
  • 3. European Humanities research– the challenges
    • Difficulty to identify and compare Humanities excellence
    • - vis- à -vis other sciences
    • - across all languages at a supra-national (European) level
    • Poor international visibility of research output in national languages
    • (considered as necessary vehicles for Humanities research)
    • Multiplicity of formats for research output (monographs, edited volumes, journals [less than 1/3 !], conference proceedings, web-based content and data, outreach)
  • 4. ERIH – the origins
    • SCH Workshop on evaluation of scientific production in Humanities (Budapest 2001):
    • Existing citation indices (AHCI, SSCI) have unsatisfactory coverage of European Humanities research
    • Need for a European citation index for the Humanities as additional tool for research assessment
    • ESF requested to compile lists of reference journals in the first instance
  • 5. ERIH – objectives / methods
    • Benchmarking tool for comparisons at aggregate level
    • Focus on format used in other sciences ( journals )
    • > achieve a degree of initial comparability
    • BUT : methodology needed for other formats
    • Encourage ’best practice’ in the publication of journals in the Humanites
    • Peer-review : discursive identification of excellence
    • > across all fields of the Humanities
    • > across Europe < national consultations
    • European Research Area :
    • Improve access to European Humanities research across all languages (basis for VLE?)
    • ERIH as a tool for the strengthening of Humanities scholarship in Europe created by scholars for scholars
  • 6. ERIH – process
    • Set up Steering Committee
    • Define the first 15 disciplines to be included
    • Define categories of journals: A, B, C
    • Compile guidelines for inter/national panels
    • Approves Expert Panels (members suggested by MO’s, SCH, StComm, in the future: ENCoPs)
    • Validates work of expert panels
    • Produce and process material
    • Input from ESF MO’s (national panels / reference tools) according to guidelines
    • Initially 15 international panels who compile categorised lists of good peer-reviewed research journals in 15 disciplines
  • 7. ERIH – current coverage
    • Index Coverage (publication criteria):
    • Peer-reviewed journals; ISSN; bibliographic data
    • for cited references; author data
    • Index coverage (discipline criteria):
    • 15 subject groups defined, some covering areas
    • between Humanities and Social Sciences
    • (anthropology; education; psychology)
    • Index coverage (internationality criteria):
    • Cat. A (<25%)/B: international journals (defined by status: reputation, readership, authors, citations),
    • Cat. C: important European journals with more regional circulation (ESF MO Europe: not Russia)
    • All journals included – whether A-B-C - to be considered good research journals
  • 8. ERIH – categories
    • Precision of criteria for international journals
    • a genuine, varied and regular international cohort of contributors and readership
    • a consistently high-quality scholarly content
    • a broad consensus within the field concerning international status and visibility, insofar as possible
    • a quality control mechanism, normally through peer-review
    • Plus some, not necessarily all, of the following characteristics:
    • Active international advisory board
    • Openness to unsolicited contributions
    • Highly discriminating and selective in the choice of articles published
    • Publication on time and to an agreed schedule
    • Degree to which these characteristics are applied could be added to the criteria distinguishing journals in categories ‘A’ and ‘B’
  • 9. ERIH – current disciplines/panels
    • ANTHROPOLOGY
    • Archaeology
    • Art and Art History
    • Classical Studies
    • Gender Studies
    • History
    • History & Philosophy
    • of Science
    • Linguistics
    • Literary Studies
    • Musicology
    • Oriental & African Studies
    • Pedagogical & Educational Research
    • Philosophy
    • Psychology
    • Religious Studies
    • & Theology
    • Possible Additions for updates :
    • Archives, Library & Museum Studies
    • Film, Media & Cultural Studies
    • General Humanities
  • 10. ERIH – towards the « initial lists »
    • 2003/04: MO’s provide input based on previous national consultation (panels; reference tools)
    • 2005/II: Expert Panels work
    • - define field / remit (“scope notes”),
    • - analyse and assess input received,
    • - suggest circles of consultation,
    • - consult where gaps are identified (eliminate / add),
    • - discuss categories (multiple listing; single listing),
    • - produce draft list, explain methods and problems.
    • 2006 (mid): Wide consultation of
    • - ESF MO’s,
    • - (European) subject associations,
    • - specialist research centres
    • 2007 / 2008: Publication of the lists in three batches
    • 2008: Update / feedback
    • Online form (quantitative information): contact publishers, editors, European subject associations, national subject associations (through ENCoPs)
  • 11. Peer review at work…
    • Layers :
    • Input: National panels / scientific communities
    • Selection: Expert Panels define scope, analyse and assess input, produce lists
    • Consultation: MOs, subject associations (European level and some national), specialist research centres
    • Calibrate/harmonise: ERIH Steering Committee
    • Feedback: open process leading to updates in 2008
    • Challenges :
    • Wide differences in quality of lists received from MOs
    • Domain-specific (e.g.: cult./soc./evol. anthropology; philosophy and ethics)
    • Language (e.g.: “forum languages” in art history; “international languages” in Oriental & African studies
    • Some panels more reluctant to overrule authority of (own) national panels
    • Outside peer pressure during consultation phases
  • 12. ERIH insights – quality through peer review?
    • Peer review recognised as the only practicable method in basic research (standard method used in scientific communications themselves)
    • Peer review can overrule weight of numbers for better (detect originality) or for worse (defend conservatism)
    • Peer review introduces comparability into discussions of national discourses in Humanities scholarship
  • 13. Evolution of ERIH Lists 1st batch of published initial lists are highlighted in yellow         Discipline Initial submission 1st draft lists Consultation 1a Consultation 1b Initial lists     Mar-Apr 2006 Spring 2006 Summer 2006 2007 Anthropology (social and evolutionary) 752 154 133 17 242 Archaeology 1310 524 290 25 419 Art and Art History 938 445 292 16 472 Classical Studies 619 256 321 11 251 Gender Studies 155 181 37 6 119 History 1419 874 508 71 907 History and Philosophy of Science 806 145 64 4 166 Linguistics 1093 680 391 34 586 Literature 1453 1481 706 10 802 Music and Musicology 204 n/a 187 4 166 Oriental and African Studies 196 588 386 14 505 Pedagogical and Educational Research 666 404 271 92 470 Philosophy 658 320 153 22 305 Psychology 1198 1201 159 4 634 Religious Studies and Theology 745 n/a 580 10 371
  • 14. ERIH – Update in 2009
    • Recomposition of Expert panels: panel rotation mechanism and inclusion of new experts, document setting out methodology of recomposition
    • Summary and Process of Methodology that underpinned work on ERIH “initial Lists”
    • Summary and Process of Methodology that underpinned work of ERIH Steering Committee
    • Panel meetings to take place from October 2008 – February 2009. ERIH Lists to be published incrementally in 2008 and 2009
  • 15. ERIH – Update in 2009
    • Feedback on ERIH “initial List” Anthropology (2007/08)
    • Of crucial importance in the updating of the “initial Lists”
    • ERIH office has been contacting Editors and Publishers of relevant journals to ask that they submit quantitative data about their journals
    • ERIH Feedback form: https://www2.esf.org/feedback.asp
    • Difficult process because no central database of contact details exists ( [email_address] etc)
    • Only 26% of journals on “initial List” have been commented upon by Editors / Publishers
  • 16. ERIH – futures?
    • Open Source
    • - ERIH is creating consensus on excellence
    •             (> priority list for inclusions)
    •             - ERIH includes publications of ESF MO’s, incl. learned societies
    •             (> institutional incentive?)
    •             - link of access / assess may create incentive to use OA
    •             [perhaps further stimulated by universities through support for IR’s]
  • 17. ERIH – futures?
    • Open source futures?
    • - ERIH aims to create a consensus on excellence
    •             (> priority list for inclusions)
    •             - ERIH includes publications of ESF MO’s, incl. learned societies
    •             (> institutional incentive?)
    •             - link of access / assess may create incentive to use OA
    •             [perhaps further stimulated by universities through support for IR’s]
  • 18. ERIH – futures?
    • Open Questions :
    •             - Measuring influence of OA journals when not cited: downloads and citation refer to different phases in research process
    •             - OA portal appropriate for a highly diverse field?
    •             - How to link OA to IR’s [additional institutional stakeholders]?
    •             - OA as linked to budget model of learned societies?
    •             - How to integrate current knowledge about delayed impact in most Humanities [6-month-rule not applicable?]?