Designing Tools for Programme Evaluation in Open and Distance Learning


Published on

Presentation at IGNOU workshop in 2007

Published in: Education, Economy & Finance
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Designing Tools for Programme Evaluation in Open and Distance Learning

  1. 1. Evaluating a Programme in Distance Education Preparing Tools for Data Collection
  2. 2. Session Objectives <ul><li>After this session, you are expected to be able to: </li></ul><ul><li>Identify appropriate tools for data collection; </li></ul><ul><li>List the characteristics of an appropriate tool; </li></ul><ul><li>Prepare tools for data collection to evaluate a programme in Open and Distance Learning. </li></ul>
  3. 3. What is Programme Evaluation? <ul><li>A systematic gathering, analysis and reporting of data about a programme to assist decision-making. </li></ul><ul><li>A process of measuring the worth of a programme. </li></ul>
  4. 4. Mesurement in Programme Evaluation <ul><li>What? </li></ul><ul><li>How? </li></ul>Learning, Impact of the programme, cost-effectiveness, opinions, etc. Require some tools; quantification of data.
  5. 5. Range of Data Collection Tools Machine Recording of data (video, computer) Standardized Tests Questionnaires Checklists Interview Schedules Direct Observation Sheets
  6. 6. Characteristics of a Tool <ul><li>Validity: </li></ul><ul><li>Reliability: </li></ul>Does the tool measure what it is supposed to measure? How consistent the tool is in measuring the phenomenon?
  7. 7. Types of Questions in a Tool <ul><li>Behaviour/ Action-based: </li></ul><ul><li>Attitude/ Opinion-based: </li></ul>What the student does? Ownership of electronic gadgets, attendance in counselling. <ul><li>Classification-based: </li></ul>What the student thinks? Writing of assignments, reading of SLM What the student is? Age, gender, and othe demographic variables
  8. 8. How Questions might be responded <ul><li>Open-ended/ Close-ended Questions </li></ul><ul><li>Types of Data and precision sought </li></ul><ul><li>Nominal/ Categorical, eg. RC/SC </li></ul><ul><li>Ordinal, eg. Rank order of choice of technology </li></ul><ul><li>Interval, eg. Likert scale, assume equal difference between two points in a scale </li></ul><ul><li>Ratio, eg. Distance of SC from home, has an absolute zero </li></ul>
  9. 9. Steps in Preparation of Tools <ul><li>Objectives of data collection </li></ul><ul><li>Collection of an item pool </li></ul><ul><li>Validation of the items </li></ul><ul><li>Pre-test for reliability </li></ul><ul><li>Modification and finalization </li></ul>
  10. 10. Issues in Tool Preparation <ul><li>Consider the language and comprehension level of your respondents </li></ul><ul><li>Be brief and focused </li></ul><ul><li>Think about the flow of the questions (logical) </li></ul><ul><li>Avoid ambiguous words </li></ul><ul><li>Avoid questions with a negative in them </li></ul><ul><li>Avoid leading questions, eg. Do you agree that assignments are necessary in Distance Education? Yes  No  </li></ul><ul><li>Avoid loaded questions, eg. Do you prefer conventional education, because there is widespread corruption in distance education? </li></ul><ul><li>Yes  No  </li></ul>
  11. 11. Issues in Tool Preparation <ul><li>Do not suggest the range of acceptable response, eg. How many hours have you spent in reading this Block? </li></ul><ul><li>(a) < 2 hours  (b) 2-5 hours  (c) 6-10 hours  (d) 11-15 hours  </li></ul><ul><li>Avoid questions that are irrelevant, eg. At what time of the day you prefer to attend counselling? </li></ul><ul><li>(a) Morning  (b) Afternoon  (c) Evening  (d) Night  </li></ul><ul><li>Do not mix more than one question, eg. How do you find the counselling sessions helpful in understanding Course 01/02/04/05? </li></ul><ul><li>(a) Excellent  (b) Good  (c) Satisfactory  (d) Unsatisfactory  </li></ul>
  12. 12. What Tools to Use? <ul><li>A Programme Coordinator wants to gather feedback every year. </li></ul><ul><li>A Course-writer/ Coordinator wants to check the instructional design quality of SLM </li></ul><ul><li>An Assistant Regional Director wants to observe a counselling session/ practical for monitoring. </li></ul><ul><li>An evaluator wants feedback on counselling sessions from counsellors. </li></ul><ul><li>A Course Coordinator wants Expert Opinion on the quality of a course. </li></ul><ul><li>An evaluator wants feedback on the quality of assignment evaluation and two-way communication. </li></ul>
  13. 13. Credits Thanks to the Participants of this Workshop and earlier workshop on Programme Evaluation Thanks to the University for giving such opportunities Thanks to the Workshop Coordinator, Prof. Basanti Pradhan A STRIDE-IGNOU PRESENTATION THANK YOU