Summary of course curriculum evaluation

  • 315 views
Uploaded on

Summary of course on Curriculum Evaluation

Summary of course on Curriculum Evaluation

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
315
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
11
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 1 MPF 1483: CURRICULUM EVALUATION Prepared by DR. SHAFEEQ HUSSAIN VAZHATHODI AL-HUDAWI (2013), Senior Lecturer, Faculty of Education, UTM Johor, MALAYSIA I. DEFINITIONS Various definitions: 1. Tyler (1950): the process of determining to what extend educational objectives are being attained 2. Borg & Gall (1983): the process of making judgment about the merit, value or worth of educational programmes, projects, materials and techniques 3. Smith & Glass (1987): the process of establishing value judgments based on evidence abt a program / product 4. Stufflebeam et al. (1971): the process of delineating, obtaining & providing useful info for judging decision alternatives 5. Provus (1971): the comparison of performance to some standards to determine whether discrepancies existed 6. The process of collecting data on a programme to determine its value or worth with the aim of deciding whether to adopt, reject, or revise the programme. 7. Joint Committee on Standards for Educational Evaluation (2003) process of systematic and focussed determination of a curriculum’s value. “Evaluation is the systematic assessment of the worth or merit of an object” (1994, p. 3) II. A BRIEF SKETCH ON EVALUATION -Evaluation is closely associated with assessing achievement of students against behavioural objectives or conducting norm-reference testing. -From 1970, emphasis was given to professional judgment, i.e., collection and analysis of quality of information for judging and deciding on a curriculum. In short, Evaluation is an integral part of the curriculum development process III. TYPES OF EVALUATION Acccording to Scriven (1967), there are two types of evaluation in curriculum: 1) Formative evaluation – an on-going program - it is a program improvement – provide data about educational program  to assist developer in improving the program For example, results of formative evaluation may help in 1. Selection of programme components 2. Modification of programme elements 2) Summative evaluation – it is done at the completion of a program - it concerned with overall effectiveness of the program - it provides data to determine the merits and demerits of the programme For example, results of formative evaluation may help in 1. Determining what were the effects of the curriculum, and evaluate them whether or not, they were intended 2. Noticing whether something was overlooked by that everyone else
  • 2. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 2 IV. GOALS OF EVALUATION IN CURRICULUM DEVELOPMENT • Enables curriculum makers reviewing and modifying – to cater the current and future needs • Curriculum should be continuously reviewed & reviewed WHY?? • To MAINTAIN, MODIFY - Quality - Relevancy - Adequacy • OR ELIMINATE QUESTIONS SHOULD BE ASKED: 1) Does the program (curriculum) meet existing or expected needs? 2) Does the program contain extraneous and outdated materials? 3) Are the students able to perform adequately once they finish their study? Wentling (1980): EVALUATION MUST DO MORE THAN : • just analyzing the extent to which a program had adhered to an original plan • OR attained its primary goals and objectives Therefore, curriculum evaluation needs to go beyond the assessment of student behavior (thus, Tyler’s definition of CE is inadequate) • It should include the overall effect on students, teachers and society. BECAUSE Curriculum is inclusive of • INTENTS (Aims, Goals and Objectives) • CONTENTS (Various subject-matter contents included, their Scope and sequence, balance; Teaching-learning materials, resources and experiences, i.e, syllabus, content outline, textbooks) • PROCESS(Pedagogy, Learning strategies) • PRODUCT or OUTCOME • Curriculum is the link between society and school • Through Curriculum we meet Educational goal of the nation • The task of evaluating the curriculum involves a COMPLEX PROCESS • Hence, CE is about evaluating EVERYTHING about curriculum. CE should have strategies to determine whether decision about all the elements of curriculum mentioned above is optimal V. THE WORKING DEFINITION OF ‘CURRICULUM EVALUATION’ Evaluation is the provision of information at… • A: STAGES of determining of aims, planning, tryout, field trail, implementation and quality control of program development, concerning… • B: ENTITY of Teachers’ guide, -study material, - instructional tools, -the whole package, from the point of view of… • C: CRITERIA of -Fit to standards, -eliciting processes, -yielding outcomes on the basis of • D: DATA such as -judgment, - observation, -examination of product summarized in • E: MODE OF SUMMARY of: Quantitative, Qualitative, Mixed Methods
  • 3. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 3 for sake of making decision about • F: ROLE: -modifying, - revising the use of the program. VI. MODELS OF ‘CURRICULUM EVALUATION’ CIPP model aims to evaluate how CORE VALUES are put in 1. GOALS (Context Evaluation) 2. PLANS (Input Evaluation) 3. ACTIONS (Process Evaluation) 4. OUTCOME (Product Evaluation) See the FIGURE in the appendix DETAILS ON CIPP MODEL (Finch & Bjorquist, 1977) 1) Context evaluation  Whether or not to offer a curriculum  If so, what its parameters will be – including goals and objectives  It includes- environment, students’ b/ground, school climate, goals and objective of the curriculum  It involves analysis of goals and needs in a specific education setting  Goals should be reviewed and revised periodically in relation to: 1) the reasons for offering the program 2) the intended target group 3) the content As curriculum developers:  Establish conducive learning environment to promote a positive learning climate  Monitor current trends and issues in education  Responsive to latest innovation and development in education EXAMPLE: Context: Examine the environment in which technology is used in teaching and learning 2) Input evaluation • Deciding what resources and strategies that will be used to achieve curriculum goals and objectives • This includes – the quality of the curriculum and syllabus, students, teachers, staff, facilities and infrastructure EXAMPLE: Input: Examine what resources are put into technology integration 3) Process evaluation • Focuses on decisions associated with curriculum effects on students • Whether the content is learned by students • Involves the data collection • Data is collected over a period of time to detect strengths and weaknesses of the program • Process evaluation includes: teachers’ methods of teaching, students’ performance / achievement, the facilities used • Conduct professional development courses to upgrade (teaching) skills of teachers • Review students assessment EXAMPLE: Process: Did the technology integration work smoothly? 4) Product evaluation • To determine the extent to which the goals of the programs have been achieved • It is also involves data collection  through feedback from former
  • 4. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 4 students or graduates, employer and society • Data collected - to make decisions about the program • Whether to retain or modify the existence program • It concerns on the number and quality of students produced • Can get from teachers and students • Info – valuable in determining adequacy and appropriateness of the curriculum EXAMPLE Product: Did the learners learn using technology? How do you know? MODEL TWO: CONGRUENCE- CONTINGENCY MODEL by Robert Stakes Here evaluation is done through Formal means of collecting for extensive data by • Observing dynamics among people involved in curriculum process • Allowing them greater participation in judging programs • Taking position regarding a program’s worth By considering 1. Antecedents: Conditions prior to teaching and learning that may influence outcomes 2. Transactions: The Process of interactions among  Students and teachers  Students and Students  Students and curriculum materials which are affected by time factors, space arrangements, communication flow, and teaching process 3. Outcomes: the Immediate and long-range program results of  Student achievement,  teachers’ perception of their competence and their influence on administrative policies Here we look at the CONSIDERED and ACTUAL Antecedents, Transactions and Outcomes. SEE the diagram in the Appendix VII. APPROACHES TO CURRICULUM EVALUATION • Approaches to curriculum evaluation mean the beliefs and assumptions (often termed 'philosophies' or 'perspectives') of designers of the curriculum on ‘what should be evaluated’? • The important approaches are: • Traditional: What are the important aspects of our cultural heritage that should be preserved, • What are the important aspects of our cultural heritage that should be preserved i.e., the students can be manipulated in order to bring about desired results’. THEREFORE, ‘the need for efficiency as seen in the manufacturing sector’ (Hopkins, 1994) • Experiential: What experience will lead to the healthy growth of the individual?, • What experience will lead to the healthy growth of the individual? i.e., learning occurred in a cycle of a) experience b) reflection c) action • Structure of Discipline: What is the structure of the disciplines of knowledge? • What is the structure of the disciplines of knowledge?, i.e., in order for members of society to function well in the wake of
  • 5. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 5 the global economy, there is a need to acquire high levels of literacy and numeric in subject areas • Behavioural: At the completion of the curriculum, what should the learner be able to do? • What is the structure of the disciplines of knowledge?, i.e., learning occurs when behavior indicates that a given stimulus was effective in producing the desired outcome • Constructivist: How can people learn to make sense of world and to think more productively and creatively? • How can people learn to make sense of world and to think more productively and creatively?, i.e., learning takes place in such a manner that the recipients of information build up on those skills and knowledge that they receive from the environment’. THEREFORE, ‘most of the time, there is a need to teach ideas in holistic manner’ VIII. ECLECTIC APPROACH TO CE Utilizing various scientific, humanistic or scientific-humanistic approaches simultaneously in order to overcome the focus of CE model on Some particular feature of evaluation Some unique functions / roles For they prescribe Specific patterns for evaluation activities Hence, they are not ALTERNATIVES, but COMPLEMENTARY to each other, and therefore ECLECTIC approach, so as to  Analyze various political, social and moral reals as to their significance in Curriculum IX. IMPORTANT CONCEPTS, TO DERIVING GENERAL AIMS AND SPECIFIC MAJOR OBJECTIVES a. Critical changes in Society a. Most Important changes, their implications on objectives b. Nature of changes: economic, social, health, development c. Employment Patterns of the time d. Requirement in Health, Welfare, Political and Social Affairs e. Students’ concerns of learning needs f. Development in the content of the Subject- matter b. Relevance and Balance c. The Language of Instruction d. New Forms of Education e. Minimum Learning Requirements • A comprehensive evaluation framework or model is necessary to achieve a systematic, effective and efficient evaluation X. SUMMARY ON SIX STAGES OF CURRICULUM EVALATION See the APPENDIX XI. STAGE ONE: Determination of General Educational Aims Curriculum evaluation is NOT’ just determining whether educational objectives are achieved, but rather it is about determining the worth and merits of objectives themselves
  • 6. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 6 XII. EVALUTION CRITERIA & PROCEDURE at the PLANNING STAGE At the PLANNING STAGE, we look how well we have selected, conceptualized and formulated 1A. Instructional Objectives 1B. Scope and Sequence of the content 1C. Teaching Learning Strategies 1D.Instructional Materials 1Aa. Instructional Objectives: CRITERIA: Are they • RELATED to the OBJECTIVE OF THE PROGRAM • CLEARLY STATED • APPROPRIATE FOR, AND ATTAINABLE by TARGET LEARNERS (based on level and mental development). • IMPORTANT ENOUGH to encourage further learning 1Ab. Instructional Objectives: PROCEDURE (How): • Seeking and Analyzing relevant opinions and judgments by panel of experts (Curriculum specialists, Psychologists, etc). 1Ba: Evaluation of the Scope & Sequence of the Content of Instruction: CRITERIA • Relevance to Instructional Objectives • Up-to-datedness of Content • Relevance to the child and his environment • Content Balance • Balance in the content in terms of students’ activities envisaged. • Balance between various subdivision of the subject matter • Organizational Structure of the content 1Bb: Evaluation of the Scope & Sequence of The Content of Instruction: PROCEDURE (How) • Seeking and Analyzing relevant opinions and judgments by panel of • Subject Specialist • Educational Psychologists • Teacher-Educators • Experienced Teachers 1Ca. Teaching-Learning Strategies: CRITERIA: • Effectiveness & Economy • Feasibility in Classroom • Teacher Competence 1Ca. Teaching-Learning Strategies: PROCEDURE (How):  In-House Analysis  Experts’ Panel  Simulation at Center  Limited Tryout in Schools 1Da. Instructional Materials: CRITERIA: • INSTRUCTIONAL CHARACTERISTICS: Contents are academically • related to instructional objectives • valid and reliable • logically Organized psychologically • relevant • appropriately Sequenced • Hierarchically structured and sequence • Reinforcing • Promoting Understanding and Critical Thinking Linguistically • Accuracy and suitability • PRACTICAL CHARACTERISTICS
  • 7. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 7 • Cost Range • Flexibility to adapt the programs to TL condition • Easiness to use 1Da. Instructional Materials Strategies: PROCEDURE (How): • Seeking and Analyzing relevant opinions and judgments by panel of • Subject Specialist • Educational Psychologists • Teacher-Educators • Experienced Teachers • Production Experts These FOUR elements of curriculum, i.e., 1A, 1B, 1C, 1D should be analyzed based on the CRITERIA (1Aa,1Ba…), following the PROCEDURE (1Ab, 1Bb,….). XIII. 5 QESTIONS OF VALUES IN CURRICULUM EVALUATION These ‘5 Q’s’ address ‘What ‘evaluation’ can do?’. The 5Q’s are The Questions of: a. INTRINSIC VALUE This related to: i. Goodness, up-to-datedness & Appropriateness of ‘planned’ and ‘enacted curriculum’ at philosophical and psychological, sociological and subject-areas levels, and ii. Their Congruence with NATIONAL EDUCATIONAL PHILOSOPHY & NATIONAL EDUCATIONAL CONCERNS How to do this (Procedure): i. Getting ‘specialists’ and ‘experts’ to analyze the curriculum document b. INSTRUMENTAL VALUE decides the ROLE of Evaluation at the sociological or pedagogical context, i.e. What is the CURRICULUM Good for?, i.e., Will what is planned in the curriculum be attained? • Objective of the program: ‘planned curriculum’ versus ‘enacted curriculum’ To what extent? • With the suggested ‘contents, materials, methods’ can the ‘philosophical and psychological’ orientations be achieved By which students? c. COMPARATIVE VALUE Is the NEW program Better than the OLD one in terms of • Skill training • Students advancement • Delivery and other costs d. IDEALIZATION VALUE How to have the BEST possible CURRICULUM? in order to HIGHTEN Students achievement, for optimal benefits. This is a continual concern to fine-tune curriculum entities: Alternatives: to involve students fully into learning? f. DECISION VALUE Making QUALITY decision on whether to RETAIN, MODIFY OR DISCARD the program based on the above FOUR considerations AND evidences and Here it is important to consider ‘WHAT IS THE VALUE OF THESE DECISIONS? XIV. NORM & CRITERION- BASED EVALUATIONS Norm-Referenced Evaluation  Evaluating students’ performance relative to the performance of other students • The performance of current students or of previous students
  • 8. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 8 can be compared, according to standards of group, where students are grouped for particular courses Criterion-Referenced Evaluation  Measuring students’ actual performance and comparing it with the objectives of instruction identified in the syllabus, or  on a specific criterion (e.g., Evaluation of mastery learning schemes) XV. EVLAUTION OF THE EFFECTIVENESS OF TEACHER TRAINING At the LARGE SCALE IMPLEMENTATION, it is necessary to provide INTRODUCTORY AND SUPPORTING COURSES (TEACHER TRAINING) a. Training related to activities and background information b. Training related to program objectives c. Training related to teachers’ role and classroom management d. Training related to criteria for monitoring students’ achievement. Furthermore, the EVALUATION OF THE EFFECTIVENESS OF SUCH TEACHER TRAINING COURSES are very significant. CRUCIAL influence of teacher in effectiveness of implementation is acknowledged, but NOT REFECTED in evaluating the effectiveness of such courses. Such course can be of various types, namely:  Concentrated workshop type  Preparatory course spread over weeks  Extended courses with specialists  Action-research  Development-Cluster groups All these types of course need to be evaluated for their effectiveness in terms of  Teachers’ knowledge of o the subject matter o activities o aims and objectives o acceptance of classroom activities with are congruent to aims  changes in Teacher’s attitude to the new program course and students behavior HOW that can be done:  Interviews  Checklist  Questionnaires  Follow-up observations  Informal discussion  Involving Teachers in devising new methods, than they receive information XVI. THE NEED FOR QUALITY CONTROL OF THE IMPLEMENTED PROGRAM At the large-scale implementation stage, curriculum tends to DETERIORATE, by being less effective. Loss of effectiveness of particular  curricula,  methods or procedures to  entire student population,  some schools,  subgroup of students. By decrease in  Achievement Level  student interest  in higher mental process (application, analysis, synthesis) Therefore, the EVALUATOR should CONTROL the QUALITY in terms of 1. Need 2. Cause 3. Corrective measures By seeing
  • 9. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 9 1. Problem 2. Process 3. Data STUDY the Appendix on Schemata of Quality Control XVII. CONTINUITY IN THE PROCESS OF QUALITY CONTROL OF THE IMPLEMENTED CURRICULUM Evaluation itself is a ‘quality control’ of  Curriculum  Instructional methods  Procedures That is, it is a continuous process of  Collecting information on problem, process,  Taking CORRECTIVE measures. It should be CONTINUED, because EDUCATION is SIGNIFICANT public enterprise, involving MILLIONS of children.  Quality control require effort, time & money but vital to detect problem, Hence we need a curriculum center to undertake quality control, CONTINUALLY and SYSTEMATICALY to make curriculum  RELEVANT  RESPOSIVE to the needs of community and nation
  • 10. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 10 APPENDIX: Different stages of program development and utilisation: different methods, strategies, personS Determination of General Aims Decision about: general aims, school structure Studies on: Expected changes, Cultural Values, Social forces, Present level of achievement, feasibility of programs Planning Writing Outline: Preparing instructional material Examining adequacy of objective, contents, strategies Judgement of material Tryout Monitoring teaching in tryout classes, Modifying material Collect evidence through observation, judgement, discussion with teachers, students Student products Field-Trail Slightly modify program , Determine optimal conditions of program use Collect evidence about the efficiency of program under various conditions Implementation Links with supervisors, examination system, teacher training Examining final form Evidence on efficiency of teacher training Quality Control Implement recommendation, plan ‘second generation’ programs Examining quality of implementation Studying reason for changes in efficiency Suggesting remedies if needed
  • 11. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 11 APPENDIX: Schemata on Quality Control Need for Quality Control Cause of deterioration Corrective measures and effectiveness Problem Does implemented curriculum remain effective Why and how deterioration originated -Appropriate quality control measures -Is particular measure effective Process Compare student achievement data in the current term with those in the previous terms or years Survey how curriculum is implemented, under what conditions, to what students groups -Analyze and compare a series of formative tests -derive hypothesis which explain why the effectiveness of the curriculum decreasing -Appropriate quality control measures -Is particular measure effective under small-scale experimental situations -apply the verified quality control measures to the target population Data -Summative achievement programs or summative achievement tests administered every year -end of course examinations -standardized achievement tests -expert and teacher judgment -attitude and interest survey -Questionnaires -Other unobtrusive measures -formative tests -school survey data - Interview and questionnaires -expert and teacher judgment -classroom observation -formative and summative tests -attitude and interest survey -Other unobtrusive measures
  • 12. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 12
  • 13. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 13 In CE, the worth could be gauged only after assessment of the graduates needs of a program. And needs of the following types Concept Definition Example Need Something that is necessary or useful for fulfilling a defensible purpose Competent, effect instruction in the basic skill areas Defensible purpose A desired end that has been legitimated consistent with a guiding philosophy, set of professional standards, institutional mission, mandated curriculum, national constitution or public policies, etc Students’ development of basic academic skills Treatment need A certain service, competent service provider, or other helping agent Competent instructors in twelfth-grade courses in math, science and language arts Outcome need Achievement or outcome required to meet a defensible purpose Students’ demonstration of proficiency in specified areas, such as twelfth- grade math, science, and language arts Assessment A systematic assessment of the extent to which treatment or outcome needs are being met Examination of students’ scores on national tests and evaluation of the involved teachers
  • 14. ADD MORE ON TRYOUT AND FIELD TRIAL STAGES AS WELL! 14 APPENDIX: Robert Stakes’s Congruence and contegency model Curriculum Development Curriculum Evaluation Considered antecedents Actual antecedents involved Considered Transactions Evaluation of Transactions Identified Outcomes Attained Outcomes Empirical contingency Empirical contingency Logical contingency Logical contingency Congruence Congruence Congruence