Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Enabling professional development by letting go of the pedagogical paradigms

Annotated slides from reflective session paper presented at the ALT Conference, 4 September 2019, Edinburgh, UK. This presentation is relevant to all learning designers, learning technologists and online practitioners navigating the literature, research and data around online learning design for professional development. It concludes with an argument for open pedagogy, that is not defined on design, but is experienced based on learner choice.

  • Be the first to comment

  • Be the first to like this

Enabling professional development by letting go of the pedagogical paradigms

  1. 1. @mattcornock Enabling professional development by letting go of the pedagogical paradigms Matt Cornock Online CPD Coordinator, National STEM Learning Centre ALT Conference, 4 September 2019, Edinburgh
  2. 2. @mattcornock Online CPD programme Managing Behaviour Science of Learning STEM Ambassadors (4 courses) STEM Careers Learning Assessment for Learning (3 courses) Maths Subject Knowledge Practical Biology (2 courses) Practical Chemistry Practical Physics Technicians (2 courses) Primary Space (2 courses) Primary Science STEM Learning’s vision is a world-leading STEM education for all young people across the UK. Through our residential, regional and online programmes, we provide professional development for teachers and technicians in primary, secondary and FE, and STEM Ambassadors, volunteers from STEM industries, who deliver activities and work with young people. We have over 20 courses, many of which have been created in the last 2 years, to form pathways of CPD. Courses are delivered on the FutureLearn platform and most are free to access.
  3. 3. @mattcornock During the 2018/19 academic year we have delivered over 50 course instances, reaching 57,000 enrolments from the UK and internationally, with our participants completing 109,000 hours of professional development. This is a huge achievement for the programme team and only possible due to the collaboration across the organisation, including the Subject Expert Team at the National STEM Learning Centre in York, our regional partners including the Science Learning Partnerships and STEM Ambassador Hubs, and our dedicated online course facilitators. Online CPD programme 50 course instances 57,000 enrolments 109,000 hours of professional development
  4. 4. @mattcornock The dilemma in the design of open online courses. learning designed vs. learning experienced
  5. 5. @mattcornock The dilemma in the design of open online courses. learning designed vs. learning experienced This presentation is a reflection on some of the ideas that have influenced how I view online learning design for open online courses, or MOOCs. These ideas characterise the type of learning that is being designed, and typify how pedagogical approaches common to other forms of face-to-face and online learning, cannot be labelled in the same way for open online courses. Throughout, I’ve been grappling with the dilemma that the learning designed is not necessarily the learning experienced. This itself started to pose existential questions to me as a learning technologist about the role of learning design, theoretical bases and learning-outcomes based activity design. As I explored the literature, I became increasingly sceptical about how particular pedagogies were being evidenced in open online courses and learning design.
  6. 6. @mattcornock Open online courses for teacher professional development ‘co-learning’ (Laurillard, 2016) ‘self-directed’(Louws, 2017) ‘sustain and embed practice’ (DfE, 2016) I first wanted to consider the form of professional development learning we wanted our learners to achieve. These are traits of professional learning, but how are professionals supported in adopting these traits? Particularly those new to the profession, at a distance, new to identifying development needs and online learning.
  7. 7. @mattcornock Our open online courses for teacher professional development structured points of interaction sharing ideas evidence-based practice ongoing reflection sustaining the impact At STEM Learning, our online professional development courses are designed with five principles. Taking place on a social learning platform, there is mentor support and ample opportunity for learners to share ideas and reflect on practice.
  8. 8. @mattcornock Range of learning needs Teaching Practical Science: Chemistry Different approaches to practicals to make these lessons more effective... New style exam questions have really exposed my students’ lack of ability to apply their learnt skills in different contexts. I want to evaluate my current practices and see which areas I could improve in general and to get ideas for relating the experiments to the real world/engaging students. I am a non-specialist and would like to improve my strategies as well as gain more confidence when conducting practical activities. Within any one course, we welcome learners from a range of backgrounds, at different stages in their career and with a range of professional learning needs. The challenge for our course design process is to create a structured course relevant to a broad audience, but allow for learners to meet their individual needs. This is the strength of online learning, in particular situated and social learning, but leads to a contradiction in learning design.
  9. 9. @mattcornock Contradictions of open online course design personal needs vs. sequence of activity individualised timelines vs. socialisation openness of access vs. self-efficacy of learners
  10. 10. @mattcornock Retention curves… tell me nothing? How do I navigate these contradictions, drawing upon the research evidence and the theoretical literature? Within the online learning literature, much emphasis is placed on the data from the learning platform. Typically this involves analysis of retention and completion metrics. But these curves, so readily produced, can only be interpreted with a broad understanding of the course, and it’s questionable to interpret learner behaviour and choices over learning from them alone. 1.01 1.03 1.05 1.07 1.09 1.11 2.01 2.03 2.05 2.07 2.09 2.11 2.13 3.02 3.04 3.06 3.08 3.10 3.12 4.02 4.04 4.06 4.08 4.10 4.12 5.02 5.04 5.06 5.08 5.10 5.12 5.14 5.16
  11. 11. @mattcornock 1 3 5 7 9 111315171921232527293133353739414345474951535557596163656769717375 Weekly retention Weekly retention is a more interesting graph. It shows much more clearly which weeks cause retention drops and points in the course where there are opportunities to hook learners back in. The graph plots visits per step against total number of learners that week. This graph actually shows a five week course. Obviously the first week is skewed by those accessing the course to make a decision about whether it is right for them; the final week is interesting as it shows learners are most selective about the steps they’re choosing across the whole week. Inferences come from an understanding of the learning design and the content of the course. Acknowledgement to D. Jennings and S. Schmoller for suggesting exploring these patterns.
  12. 12. @mattcornock Open online course success measures need to focus on outcomes. we need to let go of retention
  13. 13. @mattcornock Open online course success measures need to focus on outcomes. we need to let go of retention My view is that completion metrics don’t reflect the value of open online courses to learners with diverse needs. Why should success be measured by how learners undertake activities that may or may not be what is required to support their development. Choice is key. Completion of a course can also be tempered by platform characteristics, for example completing steps to receive a certificate. I would like to see greater emphasis in the literature on the outcomes of professional development online courses instead. In the next few slides, I explain why this is more important than focusing on course retention.
  14. 14. @mattcornock “It is not only the magnitude of data, but also the diversity of user intentions and backgrounds and the unconstrained asynchronicity of their activities that distinguish the MOOC context from conventional classrooms.” (DeBoer, et al., 2014) DeBoer, et al. (2014) talk about the way learners use open online courses differently, to meet their individual needs. The idea of ‘unconstrained asynchronicity’ wonderfully encapsulates the idea that learners will be engaging with the course when and where it works for them, doing activities flexibly and even joining the course out of sync with other learners and facilitation periods. This presents us with learning design challenges, how to ensure our learners can still undertake professional development both in terms of social and instructional pedagogies, with and without support.
  15. 15. @mattcornock Faith in the data. seeking the patterns to explain it all Like many learning designers, I too am seeking patterns in the data to explain learning taking place. In some of the research literature, conclusions are being made about how learning takes place using any data provided by a course platform. Those familiar with FutureLearn will know that the data sets available to download stop counting engagement after a certain point, so you don’t get a complete picture of learner engagement for those who join a course significantly after a deadline. The data sets also include the course team, and particularly with smaller cohorts their involvement can also affect the data patterns. This filtering, cleansing and sense-checking of the data is not always indicated in literature, but I spend a lot of time unpicking the validity of data and the correlations within.
  16. 16. @mattcornock Faith in the data. seeking the patterns to explain it all Is this faith misplaced? Numbers cannot account for ‘life gets in the way’. Measures of success from interaction data look for engagement, and do not surface the decisions not to engage. They are not measures of learning intention, choices over learning and learning outcomes. When learning design decisions are based upon successes, they stem from a misconstrued view point that comes from ‘all activities on this course must be relevant to your needs’. Non-engagement is not necessarily a failure of design. Non-engagement itself reflects a type of learning that was not designed-for, or perhaps not designed to take place within the online course. So what exactly can learning design offer?
  17. 17. @mattcornock CONTENT EDUCATOR LEARNER COHORT OUTCOME ACTIVTY I return to the model of learning design used for our online programme, the underlying principles built from activity theory (Engeström, 2001) and models of learning interaction (Anderson, 2003). We adopt an activity-based design approach, using FutureLearn’s adaptation of the ABC Learning Design framework (Young and Perović, 2015). Activities provide a means of linking content, learners, educators, their experiences, through a process of designed learning towards an outcome.
  18. 18. @mattcornock CONTENT EDUCATOR LEARNER COHORT CONTEXT OUTCOME ACTIVTY However, it’s the context in which all this sits which is crucial to professional development. Our courses are designed in a way to structure new ideas being taken into practice, reflected upon and decisions made about the impact of those practices. They are evidence based, through the academic research and insights from other practitioners. However, how a learner takes the course into their practice is through contextualisation. At the start of all our courses, learners identify their learning goal. We use the social learning platform as a space for learners to record their thinking, as much as it is for learning with their peers. Yet, the real learning is actually taking place offline, in their practice contexts, their classrooms, scaffolded by the online course.
  19. 19. @mattcornock Learning cannot exist solely online. learning sits within a practice context
  20. 20. @mattcornock Learning cannot exist solely online. learning sits within a practice context Learning, particularly professional learning, takes place across spaces. This is why I find online learning so powerful, it’s a way of bridging spaces, providing structure to contextualise understanding and furthering of practice. The data doesn’t capture this. As learning must take place across spaces, it gives rise to outcomes that I could not have anticipated, outcomes that I have not designed for, and which, perhaps my learning activities do not support. The learning outcome is influenced not just by the activity, but the context of the learner as much as anything else.
  21. 21. @mattcornock Our learners should be supported to learn and develop ‘flexible around work’ ‘sharing’ ‘try out new ideas’ ‘rethink practice’ Our courses are showing this learning across spaces. The feedback from participants, from all levels of career experience, indicates the value of our practice-linked and reflective approach, and also the flexibility of open design for learners to decide what activities to undertake.
  22. 22. @mattcornock Course rhythms and learning routines 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55 57 59 61 63 65 5 week non-specialist research course Are there patterns from data that can still inform how learners are choosing to learn on open online courses? I’m currently looking at ‘wobble’, which is the deviation from a linear path through a course and considering what design interventions may be affecting this path.
  23. 23. @mattcornock Course rhythms and learning routines 0 1 2 3 4 5 6 7 8 9 1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 3 week specialist broad subject Temporally-dependent activities, such as an educator Q&A increase the wobble, as do starts of weeks, perhaps influenced by the weekly email reminder from FutureLearn. Some courses also have more wobble than others, but is this down to the course, its design, its content, or the context of the learners?
  24. 24. @mattcornock Course rhythms and learning routines Most participants will take the course in a linear pattern. Very few participants will start elsewhere than at the beginning. My analysis suggests 10-13% of learners first access to a course is other than at the beginning. They are not using the course structure to inform decisions, but dive straight into the course content. That first page is crucial.
  25. 25. @mattcornock A social learning platform does not restrict to social learning activities 20%-30% of learners commenting. Completion patterns differ (significantly) based on: commenting, responding to others, receiving a response to comments. Most comments are not replies. My analysis also shows that 20-30% of learners comment. There is a significant relationship between completion and commenting. But, as highlighted by Swinnerton, et al. 2017, ‘an individual commenter has more opportunity to make comments the further they progress’.
  26. 26. @mattcornock A social learning platform does not restrict to social learning activities There is a dependency between variables that is sometimes overlooked in literature in the quest for evidencing a particular pedagogy. Likewise, most comments are not replies, but in the literature there are still claims social learning is happening. There is certainly a high likelihood of vicarious learning, from non-commenters picking up ideas and concepts to compare to their own practice from comments made by others, but there is again a decision to place this in a pedagogy. The same collaboratively-designed activity could well be a reflection exercise for a learner, or a stimulus for offline discussion or experimentation. A design doesn’t have to fit one pedagogy.
  27. 27. @mattcornock Planning to engage with professional development There is also a statistically significant difference in completion based upon point of joining an open course. This is also influenced by immediacy of access, whether a cohort group has formed, but again completion rate appears the metric under investigation. Difference in completion rate whether you enrol before or after the course has started. Average time difference between steps correlated with completion.*
  28. 28. @mattcornock Planning to engage with professional development Difference in completion rate whether you enrol before or after the course has started. Average time difference between steps correlated with completion.* In future, I will be looking at whether the time taken on a course has a bearing on outcomes. This can be done by comparing time differences between step views, but I am still uncertain whether completion is the right measure to use, even if there is a correlation.
  29. 29. @mattcornock Enabling professional development needs to be met 0% 20% 40% 60% 80% 100% On self On students On colleagues Overall High/medium impact Low/no impact Our course designs are working though. The impact shown, albeit from those who have accessed most of the course, shows that they bridge the learning spaces, and support impact on practice in the classroom.
  30. 30. @mattcornock Enabling professional development needs to be met 0% 20% 40% 60% 80% 100% Understanding Changed practice Course relevance Good use of time Strongly agree/agree Disagree/strongly disagree Learners are saying that the course is relevant to them and that it’s a good use of their time. The course content is fixed, the learning needs diverse, so there must be something about the facilitation and learning design that enables contextualisation.
  31. 31. @mattcornock Enabling professional development learning “Love the structure - bite-sized pieces, and very useful discussion from participants. I'm new at teaching biology, feeling much more confident about planning.” “Seeing the practical aspects of this and exploring the comments and activities suggested by this learning community has been superb.” Teaching Practical Science: Biology Our learners are sharing their views of the learning experience, and it’s interesting to see that the implicit differences in the pedagogy experienced: bite-size, discussion, confidence-building, practical, learning community…
  32. 32. @mattcornock The pedagogy can be open too.
  33. 33. @mattcornock The pedagogy can be open too. When I first started this role, I said that I had to rethink everything I thought I knew about online learning. Social constructivism and communities of practice just didn’t work the same way in open courses lasting at most 5 weeks and with many different levels of experience, knowledge and learning goals within the cohort. But as the data and success measures repeatedly show, open online courses don’t have to abide by the rules of a particular pedagogy. The pedagogy can be open.
  34. 34. @mattcornock The pedagogy can be open too. To conclude then, I am still very much steered by a particular learning design approach, as any professional is, where I make a choice about the type of learning technologist, course designer, educationalist I want to be, influenced by my own prejudices about what I believe to be good learning and the research I select to justify my position. In course design, I have a responsibility to be open about this, to explain to learners how the course has been designed a particular way. But also to acknowledge the value of learning in a different way to how the course has been designed, and to support my learners in achieving their development goals.
  35. 35. @mattcornock The pedagogy can be open too. Alongside challenging learners with the course content, I also need to be challenging learners in their approach to professional development through online courses. This includes empowering learners to make choices about what they will and won’t engage with in order to both meet their learning needs and be open to unintended learning outcomes too.
  36. 36. @mattcornock The pedagogy can be open too. Open pedagogy, or perhaps pedagogy which is ill-defined until experienced (Cornock, 2018), is a concept which has to sit alongside formal learning design. Learning design, as Conole (2015) says, provides guidance on decisions of course creation, and facilitates the discussion and process of thinking. However, it must not constrain the outcomes that can be achieved. Whilst there may be intention in a design, enabling learners to set their own intentions and be equipped to make informed choices about what learning activities will enable them to meet their development needs, not just in a single course, but with sustained and embedded professional development.
  37. 37. @mattcornock References and reading Anderson, T. (2003). ‘Getting the Mix Right Again: An updated and theoretical rationale for interaction’, International Review of Research in Open and Distance Learning, 4(2). Conole, G. (2015). ‘Designing effective MOOCs’, Educational Media International, 52(4), 239-252. Cornock, M (2018). Engagement in MOOCs by pre-prepared versus just-in-time learners. OER18, 18-19 April 2018, Bristol, UK. DeBoer, J., Ho, A.D., Stump, G.S., Breslow, L. (2014). ‘Reconceptualizing Educational Variables for Massive Open Online Courses’, Educational Researcher, 43(2), 74-84. Department for Education (2016). Standards for teachers’ professional development: implementation guidance for school leaders, teachers and organisations that offer professional development for teachers. Downes, S. (2016). ‘Personal and personalized learning’, European Multiple MOOCs Aggregator Newsletter. Available at [Accessed 13 Mar 2019] Gynther, K. (2016). ‘Design Framework for an Adaptive MOOC Enhanced by Blended Learning: Supplementary Training and Personalized Learning for Teacher Professional Development’, The Electronic Journal of e-Learning, 14(1), 15-30. Laurillard, D. (2012). Teaching as a Design Science. Abingdon: Routledge. (Specifically types of learner intervention.) Laurillard, D. (2016). ‘The educational problem that MOOCs could solve: professional development for teachers of disadvantaged students’, Research in Learning Technology, 24(1). Louws, M.L., Meirink, J.A., Van Veen, K. and Van Driel, J.H. (2017). ‘Teachers’ self-directed learning and teaching experience: What, how, and why teachers want to learn’, Teaching and Teacher Education, 66, 171-183. Swinnerton, B., Hotchkiss, S. and Morris, N. P. (2017). Comments in MOOCs: who is doing the talking and does it help?, Journal of Computer Assisted Learning, 33, 51-64. Watted, A. and Barak, M. (2018). ‘Motivating factors of MOOC completers: Comparing between university affiliated students and general participants’, The Internet and Higher Education, 37, 11-20. Young, C. and Perović, N. (2015). ABC Learning Design. UCL.