OER Impact: Towards an Evidence Base


Published on

The open education movement has achieved much in the last decade, but there remains wide acknowledgement that the impact of OER has yet to be fully understood. A suggested approach is to address this through collective approaches that collate information and present them back in an integrated way. This has some appeal, particularly in the way it matches to ideals of openness, but needs to be implemented with care.
In this presentation I critically evaluate attempts that have been made to support communication and collaboration through mapping OER. After endorsing the basic rationale for mapping evidence surrounding OER implementation I review two examples of where this has been attempted. The Open Learning Network (OLnet) Evidence Hub used the concept of Contested Collective Intelligence to inform a discourse-centric social-semantic web application that could structure the discourses of the OER community. I provide a short critique of this approach which focuses on the data model and the metadata requirements made upon users. I go on to consider the UNESCO OER Mapping Project which set out some quite specific protocols for metadata (despite never getting beyond the prototype stage). The value of a mapping approach is defended at the same time as noting that different audience will likely have very different needs in terms of evidence.

A rationale for a new, improved evidence hub is provided along with a number of design considerations and a proposal for future development. I conclude with a brief presentation of the new Evidence Hub being developed as part of the OER Research Hub (OERRH) project. I describe the ways in which our evidence model tries to overcome some of the issues which were manifest in these earlier projects, a range of different data sources, the importance of data visualization, and account for how different types of evidence might be flexibly accommodated. The final part of the session will be given over to group discussion about the idea of mapping the OER evidence base and what the OER community might want from such services.

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

OER Impact: Towards an Evidence Base

  1. 1. OER Impact: Towards an Evidence Base Dr. Robert Farrow, The Open University #oerrhub / @oer_hub
  2. 2. Evidence of OER Impact Background Rationale Design Implementation Data Visualization Future Iteration Call for Evidence / Feedback
  3. 3. Background
  4. 4. About me • Philosopher & Educational Technologist • Based at the Institute of Educational Technology (IET) Open University UK • Research Associate (OER Research Hub) • http://flavors.me/philosopher1978 • Twitter @philosopher1978 About The Open University • Europe’s largest distance learning university • Access: foremost provider to mature, disabled and working students • A world leader in technology-enhanced pedagogy • Head of the FutureLearn MOOC consortium
  5. 5. About the Project • Funded by William & Flora Hewlett Foundation for two years • Two professors lead four researchers among a team of ten • Tasked with building the most comprehensive picture of OER impact • Organised by a set of research hypotheses • Working across different educational sectors • Collaboration model • Global reach but with a USA focus • http://oerresearchhub.org
  6. 6. Collaboration Model
  7. 7. Collaboration Model
  8. 8. Rationale: Why Evidence of OER Impact?
  9. 9. Filtering data according to sector, hypothesis & polarity Framework for comparing disparate evidence types Effective evidencebased decisionmaking and advocacy Collaborative research, analysis & dissemination Openness in action: openly licensed research instruments, data
  10. 10. Research Hypotheses Keyword Performance Openness Access Hypothesis OER improve student performance/satisfaction People use OER differently from other online materials OER widen participation in education Retention OER can help at-risk learners to finish their studies Reflection OER use leads educators to reflect on their practice Finance OER adoption brings financial benefits for students/institutions Indicators Informal learners use a variety of indicators when selecting OER Support Informal learners develop their own forms of study support Transition OER support informal learners in moving to formal study Policy OER use encourages institutions to change their policies Assessment Informal assessments motivate learners using OER
  11. 11. Methodological Challenges Incommensurability Granularity
  12. 12. Previous Initiatives
  13. 13. UNESCO Mapping Project (2012) http://unescochair.athabascau.ca/oer-mapping-exercise • Mapping to raise awareness of OER • Tracking complexity, identifying major players & actions • Community building, communication and advocacy • Localised data with centralised quality control OLnet Evidence Hub (2011-2012) http://ci.olnet.org/ • Explore and debate key challenges for OER movement • Collective intelligence: raise questions, propose solutions • Sharing relevant web resources
  14. 14. OLnet Evidence Hub Data Model
  15. 15. Designing a new evidence model
  16. 16. Metadata specification Mapping locations Encouraging contribution UI / Accessibility Diverse, fragmentary evidence Filtering / navigation Polarity (+ve/-ve) Design K.I.S.S.
  17. 17. Information Architecture
  18. 18. Title • Text Copy • Text / HTML • Supports embedding of multimedia content Hypothesis Polarity Location Sector Citation • Association of evidence with hypothesis • Evidence is either +ve/-ve in relation to a hypothesis • Geotagging / GPS • School (K12) / College / HE / Informal • Academic citation • Hyperlink / URL
  19. 19. YouTube Interview Case Study Institute Metrics Academic Papers Case Study Anecdotes Survey Data Evidence FLOW
  20. 20. Implementation
  21. 21. Technical Development http://oerresearchhub.org/2013/10/04/building-an-evidence-hub-plugin-for-wordpress/ • Popular OS platform • Easy to customize • Over 27,000 existing plugins Wordpress Customization • CSV importer • Location plugin • JSON • Project code available on GitHub • Easy to link, share and engage Openness
  22. 22. tinyurl.com/oerimpact
  23. 23. Sample Entries
  24. 24. Flexible Granular Support comments, sharing Citations field
  25. 25. Visualizing the Data
  26. 26. Sankey Diagrams track the flow of evidence
  27. 27. Global Evidence Map
  28. 28. See summaries by country with click-through to evidence
  29. 29. Detailed map view
  30. 30. Explore Projects
  31. 31. Filter evidence (College evidence in USA)
  32. 32. Summaries of evidence gathered for each hypothesis
  33. 33. Future Iteration
  34. 34. Explore Survey Data Policy Map Machine curation API Integration: SurveyMonkey, Google, etc. Data Visualizations Human curation bookmarklet Data mashups using open data Exporting data sets under open license Argumentation analysis
  35. 35. Call for Evidence / Feedback
  36. 36. We want your feedback! … and your data!
  37. 37. Thanks for listening! oerresearchhub.org rob.farrow@open.ac.uk @philosopher1978