Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

20190527_Dietmar Lampert _ New indicators for Open Sciene


Published on

Presented by Dietmar Lampert (ZSI Research Policy and Development)

during the OpenAIRE workshop "Research policy monitoring in the era of Open Science and Big Data" taking place in Ghent, Belgium on May 27th and 28th 2019

Day 1: Monitoring and Infrastructure for Open Science

Published in: Science
  • Be the first to comment

  • Be the first to like this

20190527_Dietmar Lampert _ New indicators for Open Sciene

  1. 1. What to monitor? Research policy monitoring in the era of Open Science and Big Data – the what and the how Ghent, 27 May 2019
  2. 2. What to measure?
  3. 3. Who? Who says what to measure and monitor? Who does the measuring and monitoring?
  4. 4. Why?
  5. 5. What to measure?
  6. 6. source: Jean-Claude Burgelman (EC, DG Research and Innovation) Open Science Ecosystem
  7. 7. Open Science Ecosystem System
  8. 8. Development process •indicator candidates extensive literature review •extended set of indicators series of workshops •tailored set •set ready for wider scrutiny online assessment/rev iew •refined set of indicators final review •proposed set of indicators report to EC future scenarios workshop
  9. 9. Results of the process +more than 60 indicators developed +categorised into two dimensions +clustered into seven sub-dimensions +attributed responsibility to major stakeholder groups ▪ policy-makers ▪ research funders ▪ research organisation ▪ (individual) researchers ▪ publishers
  10. 10. Results of the process Dimensions (2) and sub-dimensions (7) A) the scientific process: • conceptualisation and data gathering/creation • analysis • diffusion of results • review and evaluation B) the system level: • reputation system, recognition of contributions, trust • open science skills and awareness • science with society
  11. 11. Who?
  12. 12. Conceptualisation & data gathering/creation Requirements by research funders
  13. 13. Conceptualisation & data gathering/creation
  14. 14. Diffusion
  15. 15. System level: Reputation system, recognition of contribution, trust
  16. 16. Reputation System, Recognition of Contributions – Roles in the Scientific Process Term Definition Conceptualization Ideas; formulation or evolution of overarching research goals and aims Methodology Development or design of methodology; creation of models Software Programming, software development; designing computer programs; implementation of the computer code and supporting algorithms; testing of existing code components Validation Verification, whether as a part of the activity or separate, of the overall replication/reproducibility of results/experiments and other research outputs Formal Analysis Application of statistical, mathematical, computational, or other formal techniques to analyze or synthesize study data Investigation Conducting a research and investigation process, specifically performing the experiments, or data/evidence collection Resources Provision of study materials, reagents, materials, patients, laboratory samples, animals, instrumentation, computing resources, or other analysis tools Data curation Management activities to annotate (produce metadata), scrub data and maintain research data (including software code, where it is necessary for interpreting the data itself) for initial use and later reuse Writing – Original Draft Preparation, creation and/or presentation of the published work, specifically writing the initial draft (including substantive translation) Writing – Review & Editing Preparation, creation and/or presentation of the published work by those from the original research group, specifically critical review, commentary or revision – including pre- or post-publication stages Visualization Preparation, creation and/or presentation of the published work, specifically visualization/data presentation Supervision Oversight and leadership responsibility for the research activity planning and execution, including mentorship external to the core team Project Administration Management and coordination responsibility for the research activity planning and execution Funding acquisition Acquisition of the financial support for the project leading to this publication. sources: Amy Brand, Liz Allen, Micah Altman et al. (2015): Beyond authorship: attribution, contribution, collaboration, and credit . Liz Allen et al. (2014): Credit where credit is due.
  17. 17. System level: Reputation system, recognition of contributions, and trust
  18. 18. System level: OS skills and awareness
  19. 19. System level: OS skills and awareness
  20. 20. Insights/lessons learnt Essential preconditions for indicators to work as intended: • all concerned stakeholder groups are involved in their design and evolvement • all agree on • what an indicator should measure (and what it should not) • how it should be used (and what it must not be used for) • indicators accommodate differences in research fields • can be considerable • need to be elaborated and reflected in the respective indicators • pace at which OS is being adopted • allow the emergence of new developments • new indicators need to be tested – not just discussed – before being adopted on a larger scale
  21. 21. Thank you for your attention! Contact details: Dietmar LAMPERT ZSI – Centre for Social Innovation