Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

The iterative engagement between curation and evaluation in an open research project: A utilization-focused approach

2,261 views

Published on

The iterative engagement between curation and evaluation in an open research project: A utilization-focused approach Presentation for the AVU Conference 1-3 July 2015 by Sarah Goodier

Published in: Education
  • Be the first to comment

  • Be the first to like this

The iterative engagement between curation and evaluation in an open research project: A utilization-focused approach

  1. 1. Sarah Goodier Thomas King Cheryl Hodgkinson-Williams ROER4D Project, Centre for Innovation in Learning and Teaching, University of Cape Town 2nd International Conference of the AVU 2 July 2015 The iterative engagement between curation and evaluation in an open research project: A utilization-focused approach 7/3/20151
  2. 2. Curation in research  Curation of research – usually commercial e.g. journal articles; book chapters  Move towards researching in the open:  practice of releasing interim or draft outputs, early data sets and project planning documents during the course of project activity  Change in types of research products being shared and when as well as ways of engaging with these research products
  3. 3. in the Global South  In what ways, and under what circumstances can the adoption of OER address the increasing demand for accessible, relevant, high-quality and affordable education and what is its impact in the Global South?  Researching in the open The case of ROER4D: Research on OER for Development
  4. 4. 1. Build an empirical knowledge base on the use and impact of OER in education 2. Develop the capacity of OER researchers 3. Build a network of OER scholars 4. Communicate research to inform education policy and practice 5. Curate output as open content ROER4D Objectives
  5. 5. 1. Build an empirical knowledge base on the use and impact of OER in education 2. Develop the capacity of OER researchers 3. Build a network of OER scholars 4. Communicate research to inform education policy and practice 5. Curate output as open content ROER4D Key Evaluation Areas EVALUATION Using Utilization Focused Evaluation (UFE) supported by DECI-2
  6. 6. UFE in 12 steps: 1. Assessing program readiness 2. Assessing evaluator readiness 3. Identifying primary intended users 4. Situational analysis 5. Identification of primary intended uses 6. Focusing the evaluation 7. Evaluation design 8. Simulation of use 9. Data collection 10. Data analysis 11. Facilitation of use 12. Meta evaluation Steps in the Utilization Focused Evaluation (UFE) process
  7. 7. Curation in the ROER4D project Outputs from the project: Open approach has a considerable influence on the curation strategy and how it is shaped
  8. 8. Curation in the ROER4D project Open approach has a considerable influence on the curation strategy and how it is shaped
  9. 9. Curation platforms Platform Function Capabilities Content Management System (Sakai - Vula) Long-term, private and secure storage of project document drafts, raw data and confidential/internal documentation Security Privacy Long term curation Open access repository (OpenUCT) Long-term, open access to project outputs and linked data Security Metadata Accessibility Long-term curation Curation spaces: 1. FigShare, Zenodo 2. SlideShare Short to medium-term access to: 1. Project outputs and linked data 2. Presentations Security Various levels of metadata Broader accessibility Short to medium- term curation Cloud-based Collaboration Platform (Google Docs) Collaborative creation and editing of documents; sharing of interim documents and documents under development Accessibility Ease-of-use Simultaneous creation, editing and commenting on documents
  10. 10. Evaluating curation in ROER4D
  11. 11. Extract from ROER4D evaluation plan for curation Objective 5: Curate output as open content (outputs = documents and data) Use(s) KEQ(s) Evidence What will measured To: - better curate the content. - ensure ROER4D is using the best metadata standard across outputs. 5.1.1 How well is our own research curated in order to be shared? 5.1.2 Where is the research that has been curated and shared getting the most hits? Measure against metadata standards best practice - process review; Views and downloads Measure against the e.g. Digital Curation Centre’s (DCC) standards of best practice: Checklist for data management plan, etc. Process of curation of documents and data against checklist Monitor curation spaces to gather views and downloads data e.g. From the OpenUCT publically available statistics: Views and downloads
  12. 12. Lessons Learnt: Opportunities & Challenges  Opportunities Transparency & Adaptability:  Open and inclusive team dynamics  Capitalising on new insights Reciprocity:  Connections between curation and communication work and its evaluation  Within the evaluation plan Accessibility, Adaptability & Discoverability:  Multiple platforms and outputs  Evaluation can inform which platforms are the adequate  Challenges Readiness & Adaptability :  When to be open  Evaluation needs to adapt to a changing curation strategy Cohesiveness:  Across platforms  Within the evaluation plan Vulnerability/Instability:  3rd-party platforms  Extract analytics data regularly
  13. 13. Curation platforms & evaluation plan – an example Curation Platform Platform Function Relevant KEQs Evaluation measures Uses for the evaluation results Content Management System (Sakai) Long-term, private and secure storage of project document drafts, raw data and confidential/ internal documentation 5.1.1 How well is our own research curated in order to be shared? 5.1.2 Where is the research that has been curated and shared getting the most hits? Measure against the e.g. DCC standards of best practice Available statistics: Views and downloads - better curate the content. - ensure ROER4D is using the best metadata standard across outputs. - determine if it is productive to produce multiple formats of outputs - ensure cohesion across platforms.
  14. 14. Evaluating curation in ROER4D
  15. 15. Further reading: • DCC. (2013). Checklist for a Data Management Plan. v.4.0. Edinburgh: Digital Curation Centre. Available online: http://www.dcc.ac.uk/resources/data-management-plans [Last accessed 10 February 2015]. • Hodgkinson-Williams, C. (2013). Research on Open Educational Resources for Development in Post- secondary Education in the Global South (ROER4D) - Scoping Document. Available online: http://hdl.handle.net/11427/8430 [Last accessed 10 February 2015]. • Hodgkinson-Williams, C. and Arinto, P. (2014). Open Education for a Multicultural World: A report from the Research on Open Educational Resources for Development (ROER4D) project in the Global South. Available online: http://www.slideshare.net/ROER4D/roer4-d-hodgkinson-williams-arinto-ocwc-2014- 33930519 [Last accessed 11 February 2015]. • Hodgkinson-Williams, C. and Cartmill, T. (2014). Research on Open Educational Resources for Development in the Global South: 1st Technical Report 23 June 2013 to 27 August 2014. Available online: http://hdl.handle.net/11427/9695 [Last accessed 11 February 2015]. • Monastersky, R. (2013). Publishing Frontiers: The Library reboot. Available online: http://www.nature.com/news/publishing-frontiers-the-library-reboot-1.12664#/data (Accessed 10 February 2015) • National Information Standards Organisation (NISO) (2004). Understanding Metadata. Bethesda: National Information Standards Organisation. Available online: http://www.niso.org/ publications/press/UnderstandingMetadata.pdf [Last accessed 11 February 2015]. • Patton, M. Q. (2008). Utilization-focused evaluation. California: Sage Publications Inc. • Ramirez, R. and Brodhead, D. (2013). Utilization Focused Evaluation: A Primer for Evaluators. Penang: Southbound. • Scriven, M. (1991). Evaluation Thesaurus. California: Sage Publications Inc. • Tenopir C, Allard S, Douglass K, Aydinoglu AU, Wu L, et al. (2011). Data Sharing by Scientists: Practices and Perceptions. PLoS ONE, Vol. 6, no. 6, e21101. doi:10.1371/journal.pone.0021101
  16. 16. Links and license This work is licensed under a Creative Commons Attribution 4.0 International License. Website: http://roer4d.org sarah.goodier@uct.ac.za @SarahGoodier Excluding images, screenshots and logos and/or unless otherwise indicated on content

×