Overview of CETL programme evaluation

  • 1,127 views
Uploaded on

Outlines the formative evaluation of the Centres for Excellence in Teaching and Learning programme

Outlines the formative evaluation of the Centres for Excellence in Teaching and Learning programme

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,127
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
0
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A formative evaluation of the CETL programme
    • Peter Knight and Julie Morrison, Institute of Educational Technology, The Open University.
    • On behalf of the full Lancaster-OU formative evaluation team
    • http://kn.open.ac.uk/public/index.cfm?wpid=7076
  • 2.  
  • 3. Our evaluation approach
    • Exploring theories and practices of change embedded in the CETL programme
      • We want our work to be evaluation research
    • Utilization-focused
      • We want our evaluation research to be useful.
    • Appreciative
      • We want to tell stories about what works well.
  • 4. CETL programme theory of change
    • CETL programme has an embedded theory of change that is based on the guiding principles of reward and recognition.
    • It also assumes that:
      • Excellent teaching produces excellent learning
      • Recognising individual and institutional excellence in teaching and learning promotes excellence across the sector
      • The relatively ‘light’ steer on specific designs for excellence
    • These propositions shape our evaluation
  • 5. The broad CETL strategy
    • The CETL programme is a complex national policy instrument, with a diverse range of mechanisms, to bring about:
      • Enhanced learning outcomes;
      • Change in cultures and practices of teaching and learning that constitute inspirational embodiments;
      • New institutional systems and approaches;
      • Dissemination of clusters or beacons of good practice;
      • Collaborative mechanisms;
      • The use and development of pedagogic research;
      • Strategic coherence in institutional developments.
  • 6. Evaluation implications
    • This is a formative, utilization-focused process.
    • We identify promising and effective practices ― ‘appreciative inquiry’.
    • We are interested in the tools used to build bridges from ‘excellence’ to ‘competence’ and thence to ‘new excellence’.
    • We clarify foci, explore theories of change and discuss strategic implications for the future.
    • We look for unanticipated or unintended effects.
    • We also look for multiplier effects (e.g. matched funding).
    • We explore the effects of HEFCE’s capital investment in CETLs.
    • We are interested in connective tissue.
  • 7.  
  • 8. Data sources
    • Key informant interviews
    • Case study visits (including all Northern Ireland CETLs)
    • Survey of CETL directors’ with follow-up interviews
    • Environmental scanning
    • Web-based array of visual data with evocative vignettes and exemplars
    • Local evaluators’ outputs which we will collaboratively collate
  • 9. Working with the local evaluators
    • Challenges:
      • Variation and diversity, competing commissioning loyalties, administrative burden, unstable data and the problem of validity.
    • Resources:
      • Empathy, helped by experience of previous systems and approaches.
      • Expertise in evaluation, HE, pedagogy and e-learning
    • Approach:
      • Collegial
  • 10. Possible report structure
    • 1.0 Executive Summary: One page maximum
    • 2.0 Introduction: 2.1 Structure and audience; 2.2 Connection with other reports;
    • 2.3 Purpose; 2.4 Overview of the evidential base
    • 3.0 Brief description of the aims and scope of the CETL: 3.1 Purposes; 3.2 Specific goals; 3.3 Activities; 3.4 Roles and participants
    • 4.0 Evaluation framework and approach: 4.1 RUFDATA or equivalent statement about overall approach; 4.2 Reflections on the evaluation process
    • 5.0 Findings addressing key evaluation foci from RUFDATA or equivalent: 5.1 Student experience; 5.2 Connections with external partners; 5.4 Internal strategic impact; 5.5 Effects on teachers; 5.6 Effects on learning designs
    • 6.0 Lessons learned and future adjustments: 6.1 Overview of new knowledge resources from students; 6.2 Emerging teaching practices; 6.3 Implications for University systems and practices; 6.4 Any sector wide multiplier effects; 6.5 Adjustments and future plans; 6.6 Reflections on the idea of a CETL as a change strategy
    • Figures, appendices, data sources and references
  • 11.  
  • 12. Overall challenges
    • Staying creative
    • Insisting that we are taking a systems perspectives
    • Making the evaluation participative
      • Making it safe for colleagues to share ideas and experiences offers
      • Creating open discussion about programme evaluation
    • Working appreciatively with diversity and creativity
    • Supporting and benefiting from local evaluations
    • Doing so much in so short a time
    • Exploring the CETL programme’s embedded theory of change
  • 13.  
  • 14. Please contact us
    • [email_address]
    • [email_address]
    • http://web.mac.com/knightpeter/iWeb/Site/Welcome.html
    • http://iet.open.ac.uk/pp/peter.knight/research.cfm
  • 15. The academic team World-class in educational, programme and policy evaluation Track record of evaluative research of the change process Research-based expertise in policy support and development in Higher Education A body of published work on Higher Education policy, change, teaching and learning Professor Paul Trowler Lancaster University Professor Murray Saunders Lancaster University Dr Paul Ashwin Lancaster University Dr Sadie Williams Lancaster University Professor Peter Knight The OU Dr Joan Machell Lancaster University
  • 16.