Programme Evaluation: Understanding the design

The Big Question… How to do programme evaluation?

Evaluation is not to prove, but to improve “ ”

Programme evaluation is not course evaluation…

A systematic gathering, analysis and reporting of data about a programme to assist decision-making. What is programme evaluation?

A systematic gathering, analysis and reporting of data about a programme to assist decision-making.

Programme evaluation is not end of event exercise…

A Programme Evaluation Framework Focus Methods Tools Analysis Reporting Purpose Stakeholders Areas: COAT Time Data Collection Plan Qualitative Quantitative Sampling Checklists Questionn- aires Interview schedules Observation sheets Pre-test Data Colle- ction Analysis Interpretation Action Plan Reporting Questions Blueprint Tools Findings Decisions

Example of Focus Purpose: To improve the operation of the existing programme To assess the impact of the programme To maintain and assure quality To decide on continuing with the programme Any other?

Purpose:

To improve the operation of the existing programme

To assess the impact of the programme

To maintain and assure quality

To decide on continuing with the programme

Any other?

Example of Focus Stakeholders: Students Academic Counsellors Faculty Subject Matter Experts Employers Study Centre staff/ support staff Society

Stakeholders:

Students

Academic Counsellors

Faculty

Subject Matter Experts

Employers

Study Centre staff/ support staff

Society

COAT Questions Components: What are the components of the programme? Need to cover all… Outcomes: What the programme is intended to achieve? Activities: What activities are performed during the programme delivery? And how are they organized? Target Groups: For whom the programme is designed, and what are their needs?

Components: What are the components of the programme? Need to cover all…

Outcomes: What the programme is intended to achieve?

Activities: What activities are performed during the programme delivery? And how are they organized?

Target Groups: For whom the programme is designed, and what are their needs?

Example of Focus Components: Print materials Support services Academic counselling Electronic resource support Assignment and interactions Project work supervision Practical facilities Post qualification placement Any other?

Components:

Print materials

Support services

Academic counselling

Electronic resource support

Assignment and interactions

Project work supervision

Practical facilities

Post qualification placement

Any other?

Example of Focus Outcomes: (eg. B. Ed.) To increase the number of trained teachers in the country To improve the quality of teaching-learning at school level Any other?

Outcomes: (eg. B. Ed.)

To increase the number of trained teachers in the country

To improve the quality of teaching-learning at school level

Any other?

Example of Focus Activities: Entrance Test Admission Supply of learning materials Academic counselling Practice teaching Workshop Assignments Term-end evaluation

Activities:

Entrance Test

Admission

Supply of learning materials

Academic counselling

Practice teaching

Workshop

Assignments

Term-end evaluation

Example of Focus Target Group: In-service teachers with 2 years experience of teaching in secondary schools Needs of the target group addressed?

Target Group:

In-service teachers with 2 years experience of teaching in secondary schools

Needs of the target group addressed?

To conclude… Knowing is not enough; we must apply. Willing is not enough; we must do. -- Goethe

The Question still remains… HOW