Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Critical issues in contemporary open education research


Published on

This presentation outlines some key considerations for researchers working in the fields of open education, OER and MOOC. Key lines of debate in the open education movement will be described and critically assessed. A reflective overview of the award-winning OER Research Hub project will be used to frame several key considerations around the methodology and purpose of OER research (including 'impact' and 'open practices'). These will be compared with results from a 2016 OER Hub consultation with key stakeholders in the open education movement on research priorities for the sector. The presentation will conclude with thoughts on the potential for openness to act as a disruptive force in higher education.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Critical issues in contemporary open education research

  1. 1. Critical issues in contemporary open education research Digital University Network Seminar Society for Research into Higher Education Nov 2016 Dr. Rob Farrow @philosopher1978
  2. 2. @philosopher1978 @oer_hub
  3. 3. • Award-winning research into open education • Strategies for building worldwide open education research capacity • Available for research & consultancy (short & long term) • Current projects include:
  4. 4. Project Co-PILOT
  5. 5. Keyword Hypothesis Performance OER improve student performance/satisfaction Openness People use OER differently from other online materials Access OER widen participation in education Retention OER can help at-risk learners to finish their studies Reflection OER use leads educators to reflect on their practice Finance OER adoption brings financial benefits for students/institutions Indicators Informal learners use a variety of indicators when selecting OER Support Informal learners develop their own forms of study support Transition OER support informal learners in moving to formal study Policy OER use encourages institutions to change their policies Assessment Informal assessments motivate learners using OER
  6. 6. • Research instruments applied consistently across collaborations: surveys, interview questions, focus groups, etc. • Supplemented by integration of secondary research • ‘Agile’ research, sprinting • Thematic and methodological cohesion provided by research hypotheses Research Process
  7. 7. Geographical spread of research participation
  8. 8. Some key findings
  9. 9. • 37.6% of educators (n=268) agree or strongly agree that OER use increases student satisfaction • 27.5% of educators (n=196) agree or strongly agree that OER use improves student grades • Impact appears to be greater for non-grade related aspects: - 36.2% (n=254) OER improves student engagement ✓ - 36.2% (n=254) OER promotes new ways of learning ✓ - 35.2% (n=256) OER increases student interest in subject ✓ - 35% (n=249) OER leads to student self-reliance ✓
  10. 10. Community College Educators • Responses from CA, TX, VA, FL, TN • Most respondents were experienced teachers with postgraduate degrees • A majority teach full-time and are involved in online instruction • Around half have adapted/used OER but only around 25% create or upload OER
  11. 11. 6% 13% 14% 14% 16% 18% 19% 21% 22% 23% 3% 5% 1% 3% 4% 2% 2% 2% 2% 2% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% I make use of a wider range of multimedia I reflect more on the way that I teach I have broadened my coverage of the curriculum I now use OER study to develop my teaching I have improved ICT skills I more frequently compare my own teaching with others I have a more up-to-date knowledge of my subject area I use a broader range of teaching and learning methods I collaborate more with colleagues I make more use of culturally diverse resources strongly agree– agree– neither agree nor disagree– disagree– strongly disagree– Impact of OER use on teaching practice (community college educators n=136)
  12. 12. • 55.7% (n=370) of formal students agree or strongly agree that OER increases student satisfaction • Formal learners reported that increased interest in subject was the main outcome from using OER (60.1% n=398) • Others included increased experimentation (49.4% n=398) and gaining confidence (48.6% n=322) • For some cohorts (e.g. Saylor Academy) more than half of learners believed that they grew more confident, became interested in a wider range of subjects and felt their learning experiences improved
  13. 13. Textbook access
  14. 14. Access to high quality content Community building
  15. 15. 8.5% 8.6% 8.7% 11.5% 11.8% 12.3% 14.3% 15.2% 15.2% 15.4% 15.5% 17.1% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% ... increases interest in the subjects taught ... builds confidence ... allows me to better accommodate learners' needs ... increases collaboration and/or peer-support ... increases enthusiasm for future study ... increases participation in class discussions ... increases satisfaction with the learning experience ... leads to improved student grades ... increases engagement with lesson content ... develops independence and self-reliance ... leads to interest in a wider range of subjects ... Increases experimentation with ways of learning Strongly agree Agree Neutral Disagree Strongly disagree Perceptions of impact of OER on students (community college educators n=136)
  16. 16. 7.9% 10.8% 12.2% 18 % 23% 25.2% 25.9% 29.5% 32.4% 34.5% 37.4% 38.1% 38.8% 64.7% 69.8% Data sets Whole course Interactive games Infographics Audio podcasts Learning tools, instruments and… Lesson plans E-books Tutorials Elements of a course (e.g. a… Open textbooks Quizzes Lectures Images Videos Multimedia content is around twice as popular as other OER (including textbooks) Few reported using a whole course of OER, suggesting they cherry pick resources as needed Very few are using openly available data to teach OER Types Used (community college educators n=136)
  17. 17. 1% 2% 4% 5% 9% 11% 13% 17% 20% 26% 26% 32% 35% 43% 71% 0% 10% 20% 30% 40% 50% 60% 70% 80% OER Repositories Used (community college educators n=136)
  18. 18. Strong evidence for savings when replacing textbooks with open versions Evidence for other forms of cost savings is less clear
  19. 19. Has your institution saved money through OER? yes 44% no 19% don't know 37% “OER resources are old-school, low- tech modules that are not peer- reviewed or nationally normed. There are some interesting tentative attempts at creativity but much of OER is cr*p.” “Students return for additional classes.” “They don't like losing the revenue stream from the bookstore.” “I know that some instructors are only using OER which provides substantial savings for our students.”
  20. 20. Have your students saved money through OER? yes 62% no 13% don't know 25% “Saving money is a big incentive for students and institutions.” “I still use publishers' textbooks in my classes. I use OER as supplements to the textbook.” “My students tell me and enrollment in my classes has continually increased over those of my peers.” “I developed an online textbook for the personal health class that I teach. This saves each of my students approximately $100.” “I know that some instructors are only using OER which provides substantial savings for our students.”
  21. 21. How important is open licensing (e.g. CC)? crucial 20% very important 34% somewhat important 19% neither important nor unimportant 22% not at all important 5% More than half feel that open licensing is crucial or very important, but far fewer actually practice it. This could indicate either 1) that educators are not confident about licensing their work or 2) they feel it is an avoidable addition to their workload.
  22. 22. • Most of the respondents have used some sort of OER, though only around a quarter create OER • Most report positive effects on their teaching practice as a result of OER use, particularly around peer collaboration and improved subject knowledge • A smaller proportion (but still in excess of 40%) feel that OER use directly leads to improved reflection on pedagogical practice • Positive effects were also identified for learners, especially around increased self- reliance, subject interest and experimentation • There were similar numbers who thought OER wasn’t making much of a difference and a core of what might be termed ‘anti-OER’ responses Community college (educators) Community College Educators: Summary of Results
  23. 23. Community College Educators: Summary of Results • There were mixed views about whether OER was saving institutions money, but approximately 2/3 felt that students had saved money • Around 1/3 believe that OER is improving student attrition while around 1/2 believe it is not having an effect • There is a core of advocates who understand and actively promote OER; they adopt open educational practices and believe it leads to benefits • Only around half of OER creators have used open licensing
  24. 24. Farrow, R., de los Arcos, B., Pitt, R., & Weller, M. (2015). Who are the Open Learners? A Comparative Study Profiling non-Formal Users of Open Educational Resources. European Journal of Open, Distance and E- learning, 18(2).
  25. 25. 0% 5% 10% 15% 20% 25% 30% 35% Under 15 15-18 19-24 25-34 35-44 45-54 55-64 65-74 Over 75 Saylor (n=2299) iTunesU (n=103) OpenLearn (n=725) Age profiles by repository (N=3127)
  26. 26. iTunesU channel users were much more likely to be younger and were mostly male. They are often in full time education and use OER on an informal basis outside of their formal studies to pursue interest in a wide range of subjects.
  27. 27. Saylor Academy users are more likely to be in employment and already in possession of a degree. They tended to be middle aged and primarily motivated by professional development.
  28. 28. OpenLearn users were more likely to be older, retired, and female, and had a higher proportion of users who were motivated mainly by personal interest (though 40% are in full time employment).
  29. 29. Patterns of OER repository use (N=2460) 0% 20% 40% 60% 80% 100% Saylor (n=1802) iTunesU (n=104) OpenLearn (n=554)
  30. 30. • Data about prior qualification supports idea that OER mainly used by already educated • Most learners use a small number of repositories and indicate little awareness of OER as concept/method • Very high degree of satisfaction with OER across all repositories • However, this doesn’t necessarily equate to increased likelihood of formal study
  31. 31. Impact of OER repository on likelihood of future study Repository More likely to study formally Less likely to study formally Saylor (n=1858) 19.8% 19.9% OpenLearn (n=583) 31.4% 13.9% iTunesU (n=94) 23.4% 25.5%
  32. 32. Adaptation of open resources is high (79.4%) for all roles: This is higher than found in previous studies (e.g. Wiley, 2009) but more research is needed into adaptation behaviours (motivations, goals, techniques, evaluation, etc.) … can one adapt without altering?
  33. 33. OER Evidence Report 2014 OER Data Report 2015
  34. 34. Open Research (2016)
  35. 35. Thoughts on research challenges OER awareness is low, but OER use is high Evidence about OER impact is mixed Efficacy studies for open textbooks have become the standard form of study Lack of triangulation reinforced by nature of openness Incommensurability & granularity of evidence Openness also provides opportunities for innovating research practice
  36. 36. Thoughts on openness Openness as constellation Openness as enabler of critical pedagogies The ‘return’ of ideology into open education movement? Ethical, social and political commitments of openness
  37. 37. Deimann, M. & Farrow, R. (2013). Rethinking OER and their use: Open Education as Bildung. International Review of Online and Distance Learning 14(3). 70/2542 Farrow, R. (2016).“Constellations of Openness” in Deimann, M. and Peters, M. A. (eds.) The Philosophy of Open Learning: Peer Learning and the Intellectual Commons. New York: Peter Lang. Farrow, R. (2016). A Framework for the Ethics of Open Education. Open Praxis, 8(2). Farrow, R. (2015). Open education and critical pedagogy. Learning, Media and Technology. DOI: 10.1080/17439884.2016.1113991 Some relevant publications
  38. 38. The Open Research Agenda
  39. 39. What is the Open Research Agenda? Simple consultation and sharing exercise Invitation to share thoughts about research priorities in open education Results are discussed at conferences and workshops That discussion then becomes part of the following presentation Promoting an open research culture The hope is that we will become better able to anticipate research needs and funding opportunities as well as find potential collaborators
  40. 40. What is the Open Research Agenda?  Community consultation exercise to better understand research priorities of practitioners Intended to identify patterns across countries and stakeholders ‘ Flipping’ the conference format  Stealing research ideas Setting the agenda ourselves
  41. 41. Survey Methodology Country (main residence) Self-perception of OER expertise (adapted from Dreyfus & Dreyfus (1980) model of skill acquisition) Role (choose from list or free text) Context of work/study (based on International Standard Classification of Education (UNESCO, 2011)) Perception of research priorities (free text) Most important research questions (free text)
  42. 42. As at 31 October 2016: • 91 survey responses • Respondents from 24 countries across 5 continents: Australia, Bangladesh, Brazil, Canada, Chile, Croatia, Ecuador, Ethiopia, Germany, India, Ireland, Macedonia, Malta, Mexico, New Zealand, Portugal, South Africa, The Netherlands, United Kingdom • Approximately a further 90 participants in discussion (3 expert meetings; 3 conferences; 1 webinar) Survey Sample Overview
  43. 43. Hewlett OER Meeting (February)
  44. 44. Action Lab @ OE Global, Krakow (April)
  45. 45. Action Lab @ OE Global, Krakow
  46. 46. OER16, Edinburgh, UK (April)
  47. 47. Global OER Graduate Network (May)
  48. 48. Computers and Learning Research Group Annual Conference (June)
  49. 49. European OER Experts Meeting (June)
  50. 50. OERu Expert Meeting (October)
  51. 51. Open Education 2016 (November)
  52. 52. Summary of Results
  53. 53. Role
  54. 54. • One third of respondents described themselves as having only one role • More than three quarters (77.3%) of all respondents described themselves as wholly or in part an educator (n=75); Over 40% (n=42) said they were both an educator and an advocate • Three quarters of respondents who describe themselves in whole or part as a policymaker also describe themselves as an advocate of OER (n=11) • Some people said they had as many as 7 simultaneous roles • Most participants who responded with ‘Other’ told us their role was wholly, or in part, as a librarian (3) or researcher (13) Role
  55. 55. Expertise/Experience
  56. 56. What are the most important areas for open education research over the next year?
  57. 57. Most popular themes 0 2 4 6 8 10 12 Assessment Awareness & Perceptions Business models Professional development Quality OER for development Collaboration Evidence Impact Technology Case studies Adoption Pedagogy OEP
  58. 58. ‘Tier 2’ themes 0 0.5 1 1.5 2 2.5 3 3.5 Adult education Advocacy Credentialing, Professional development, Reuse Digital/information literacy Discoverability Implementation Improving access Policy, Collaboration Copyright & intellectual property Discourse analysis/perceptions Reuse
  59. 59. Least popular themes Accessibility Ethics Formal recognition of OER Global North/South Innovation (marginalised voices) Lifelong learning MOOC Non-formal learning OER Creation Open Data Social mission of open education + all the others not mentioned…
  60. 60. Open practices and pedagogies • OER for lifelong learning and professional development • Policy development • Benefits of ‘non-disposable assignments’ • Local/National platforms for teachers • Networked learning • OER for language development • “Open pedagogy” • Citation and cataloguing protocols • Integration of OER into curriculum • Creating assessment tools • Micro-credentialling • Recognition of prior learning • Sustainable business models
  61. 61. OER impact research • Measuring the influence of the use of OER and OEP on learning outcomes and institutions • More empirical studies with wider focus than purely on open textbook efficacy • A wider range of research methods (including qualitative approaches) • Quantifying the return on investment from going open • Scant mention of student cost savings
  62. 62. Promoting adoption and adaptation • Factors affecting adoption of individual OER • Qualitative descriptions of adaptation • Barriers to remix and sharing (& implications for design) • Barriers to institutional adoption • Strategies for easing implementation by faculty
  63. 63. Collaboration and community • Peer review • Co-operation between institutions • Transnational co-operation • Collaborative instructional design • Promoting equality of opportunity • Partnerships that provide added value • Quality
  64. 64. Technology and infrastructure • Sustainable OER ecosystem • Ways of cataloguing and sharing OER (e.g. shared database) • Linked open data for education • Open digital badging • Geographical hubs for sharing OER • Using OER with VLEs • Beyond repositories • Interoperability
  65. 65. Ethical issues • Privacy, security and trust in online learning • Ownership of (student) data and ‘safe spaces’ • Access & accessibility vs. remixable resources • Appropriate literacies • Being ethical ‘in the open’ • Some conflation with ethical issues in e-learning more gnenerally
  66. 66. Openness • Tension between differing interpretations of openness • Social and political aspects of open education • Articulating the benefits of open • Building open communities and supporting cross-community exchange
  67. 67. What are the most pressing questions that need to be answered?
  68. 68. Reflections on this approach It’s quite hard to get people to do anything! Envisaged breakdowns of role, level of expertise have been problematic because of multiple roles and most identifying as expert
  69. 69. “Messes are complex, multi-dimensional, intractable, dynamic problems that can only be partially addressed and partially resolved.” (Brydon-Miller et al., 2003:21)
  70. 70. Action Research Originally conceived by Luwin (1946) “a cycle of posing questions, gathering data, reflection, & deciding on a course of action” (Ferrance, 2000) Several approaches (traditional, contextual, radical, educational) Simple Action Research Model (MacIsaac, 1995)
  71. 71. Participatory Action Research (Freire) Blackall & Hegarty (2012:69) identify four key aspects to Participatory Action Research (PAR) which are relevant to the worldwide OER community: The explicit aim to engage all stakeholders […] in describing the problems Asking those stakeholders to research the problem and propose solutions Empowering those stakeholders to carry out their plans Repeating the cycle, reflecting on lessons learned and publishing the research
  72. 72. Suggestions for future research Holistic case studies which include a focus on efficacy and cost savings but also provide (balanced) qualitative data regarding the impact on persons and practices Detailed case studies of open pedagogy (including theoretical perspective) Studies of the narrative on ‘open’ (e.g. discourse analysis) – there are ever more voices in this space with differing interpretations Business analysis -> generate sustainable business models How can a more holistic OER ecosystem be achieved? OER World Map? Sociology/psychology of conversion, persuasion, and culture change Sharing best/effective practices: institutions, educators, learners
  73. 73. Tensions in the overall picture Desire for control and predictability Desire for freedom, exploration and innovation Advocacy ‘Pure’ research A well defined community of practice An ‘open’ community ‘I have this problem and I need a solution…’ ‘I think research should be done in…’ Local context Global context Pragmatic Ideological
  74. 74. To conclude: The evidence base about the impact of OER continues to grow Much of the debate in the USA is about the relative efficacy of OER alternatives Openness presents challenges to researchers along several axes: lack of clarity; validity; data collection; dissemination. Openness is often contextual and situated: it can be hard to generalise Certain tensions can be determined in the open education movement (conservative/radical; idealist/pragmatic; &c.) Openness can be seen to have a close connection with radical, critical connections through the association with enhanced freedom/autonomy
  75. 75. Additional image credits "Life is Sharing" (CC-BY 2.0 Generic, Alan Levine) “Open” Chuck Coker,_Virginia_downtown.jpg CC-BY-SA 3.0
  76. 76. • Award-winning research into open education • Strategies for building worldwide open education research capacity • Available for research & consultancy (short & long term) • Current projects include: