PowerPoint summary of CETL evaluation plan


Published on

This is the presentation we made to HEFCE. It summarises our plans for the evaluation of the CETL programme.

Published in: Economy & Finance, Education
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

PowerPoint summary of CETL evaluation plan

  1. 1. <ul><li>The national evaluation of the CETL programme 2005-2010: first formative phase </li></ul>CSET in the Department of Educational Research Lancaster University, The Institute of Educational Technology The Open University
  2. 2. The theory of change embedded in the CETL programme <ul><li>The following propositions flow from an embedded theory of change that is based on the guiding principles of reward and recognition </li></ul><ul><ul><li>Excellent teaching produces excellent learning </li></ul></ul><ul><ul><li>Recognising individual and institutional excellence in teaching and learning promotes excellence across the sector </li></ul></ul><ul><ul><li>The relatively ‘light’ steer on specific designs for excellence </li></ul></ul><ul><li>These propositions form the central core of concerns for the evaluation </li></ul>
  3. 3. Overall understanding of the CETL strategy <ul><li>The CETL programme is a complex national ‘policy instrument’ . </li></ul><ul><li>It incorporates a diverse range of ‘mechanisms’ . </li></ul><ul><li>They are embodied in the specific designs of individual CETL activity. </li></ul><ul><li>They are intended to bring about: </li></ul><ul><ul><li>Enhanced learning outcomes, </li></ul></ul><ul><ul><li>Change in cultures and practices of teaching and learning that constitute inspirational embodiments, </li></ul></ul><ul><ul><li>New institutional systems and approaches, </li></ul></ul><ul><ul><li>Dissemination of clusters or beacons of good practice, </li></ul></ul><ul><ul><li>Collaborative mechanisms, </li></ul></ul><ul><ul><li>The use and development of pedagogic research, </li></ul></ul><ul><ul><li>Strategic coherence in institutional developments. </li></ul></ul>
  4. 4. Implications for the evaluation <ul><li>We will focus on enclave effects, bridgeheading strategies, the development of new embedded practices and systemic change </li></ul><ul><ul><li>This is a formative, utilization-focused process. </li></ul></ul><ul><ul><li>We are interested in bridging tools that are used. </li></ul></ul><ul><ul><li>We offer a strategic commentary on provisional stabilities. This involves clarifying foci, exploring theories of change, plenty of interim reporting and discussing strategic implications for the future. </li></ul></ul><ul><ul><li>We look for unanticipated or unintended effects. </li></ul></ul><ul><ul><li>We also look for multiplier effects in resource procurement for T&L developments (e.g. matched funding). </li></ul></ul><ul><ul><li>We will explore the effects of HEFCE’s capital investment in CETLs. </li></ul></ul><ul><ul><li>We identify promising and effective practices -- ‘appreciative inquiry’. </li></ul></ul><ul><ul><li>We are interested in ‘connective tissue.. </li></ul></ul>
  5. 5. Data sources <ul><li>Key informant interviews </li></ul><ul><li>Case study visits (including all Northern Ireland CETLs) </li></ul><ul><li>Survey of CETL directors’ with follow-up interviews </li></ul><ul><li>Environmental scanning </li></ul><ul><li>Web-based array of visual data with evocative vignettes and exemplars </li></ul><ul><li>Local evaluators’ outputs which we will collaboratively collate </li></ul>
  6. 6. Working with the local evaluators <ul><li>Challenges: </li></ul><ul><ul><li>Variation and diversity, competing commissioning loyalties, administrative burden, unstable data and the problem of validity. </li></ul></ul><ul><li>Resources.: </li></ul><ul><ul><li>Empathy, helped by experience of previous systems and approaches. </li></ul></ul><ul><ul><li>Expertise in evaluation, HE, pedagogy and e-learning </li></ul></ul><ul><li>Approach: </li></ul><ul><ul><li>Participative design, agreed reporting themes, meta–review matrix </li></ul></ul>
  7. 7. Overall challenges <ul><li>Creating valid indicators of the embedded theory of change </li></ul><ul><li>Working with multiple and diverse CETL designs </li></ul><ul><li>Enabling the effective use of local evaluations </li></ul><ul><li>Doing so much in so short a time </li></ul>
  8. 8. The Evaluation Team World-class in educational, programme and policy evaluation Track record of evaluative research of the change process Research-based expertise in policy support and development in Higher Education A body of published work on Higher Education policy, change, teaching and learning Professor Paul Trowler Lancaster University Professor Murray Saunders Lancaster University Dr Paul Ashwin Lancaster University Dr Sadie Williams Lancaster University Professor Peter Knight The OU Dr Joan Machell Lancaster University