Pogram Evaluation.pptx

Curriculum and
Program Evaluation

Evaluation: What and Why?
What is evaluation and why is it important?
Attempt of an answer:
Evaluation is the systematic acquisition and assessment of information to
provide useful feedback about some object.
In an organizational or educational context, evaluation is a more delicate
process that raises a series of difficult or contentious issues.

Some terms and concepts related to
Evaluation:
B- Political issues in Evaluation:
• guiding or influencing government policy
• serving executive and legislative decision makers who make decisions about
programs
A- Stakeholders:
Individuals and groups who have a direct interest in and maybe affected by the program being evaluated
and the evaluation’s results. They are:
1- people with authority over the program: funders, policy makers…
2- people with direct responsibility for the program: administrators, managers…
3- People who are beneficiaries of the program: students, families, communities…
4- People disadvantaged by the program (those who will lose funding because of it)

C- Ethical issues in Evaluation:
1- Challenges in the contracting phase:
 Stakeholder has already decided what the findings "should be" or plans to
use the findings in an ethically questionable fashion.
 Stakeholder declares certain research questions off-limits in the evaluation, despite their
substantive relevance.
 Legitimate stakeholders are omitted from the planning process.
2- Ethical concerns regarding confidentiality or disclosure agreements:
 Disputes or uncertainties concerning ownership/distribution of the final
report, raw data…
 Although not pressured by stakeholders to violate confidentiality, the
evaluator is concerned that reporting certain findings could represent
such a violation.
 Evaluator is pressured by stakeholder to violate confidentiality.

C- Ethical issues in Evaluation:
3- Challenges in presenting findings:
 Evaluator is pressured by stakeholders to alter presentation of findings.
 Evaluator is reluctant to present full findings for unspecified reasons.
 Evaluator has discovered behavior that is illegal, unethical, dangerous…
 Evaluator is unsure of his or her ability to be objective or fair in presenting
findings.
4- Ethical concerns after the report is complete concerning misinterpretation
or misuse:
 Findings are suppressed or ignored by the stakeholder.
 Unspecified misuse by the stakeholder.
 Findings are used to punish someone (the evaluator or someone else).
 Findings are deliberately modified by the stakeholder prior to release.
 Findings are misinterpreted by the stakeholder

Program Education in Academia:
Increasing interest of colleges in Program Education: Michigan State University offers a
graduate online degree in Program Education
Part of the career opportunity statement for this major stipulates that:
“Program evaluators systematically investigate the quality of programs of all kinds and all sizes, including:
educational and school-based programs.
recreational programs.
charitable programs.
human service programs.
health-care programs.
government programs.
Program evaluators study programs using tools from many different research traditions in order to figure
out if programs are working and how programs might work better.”
http://progeval.msu.edu/

Curriculum Development Process in a
nutshell
Target
Audience
(WHO?)
Intended
outcomes
Content
Methods
Evaluations
Needs for
curriculum
identified
(WHAT?)
Curriculum
development
team
Pilot and
Field
Testing
Leads to
Makes
decisions
about
Evaluation
Redesign as necessary
Creates a draft
curriculum
product

Curriculum Development Phases and Steps in a
nutshell
I- PLANNING
1- Identify need/issue
2- Form Curriculum
development team
3- Conduct Needs
Assessment and Analysis
II- CONTENT AND
METHOD
4- State Intended
Outcomes
5- Select Content
6- Design Learning
Experiences and
Assessment
III- IMPLEMENTATION
7- Produce curriculum
product
8- Test and Revise
curriculum
9- Implement curriculum
EVALUATION:
- Design Evaluation
strategies (Formative
and Summative)
- Reporting to
stakeholders

Curriculum Development Phases and Steps
briefly explained
I- PLANNING
Identify Issue/Need
Ask questions to
define a need or
problem and
consequently select
curriculum team
Form Curriculum Team
Roles and functions
are defined as well as
principles of
collaboration and
teamwork
Conduct Needs Assessment and
Analysis
Assessment:
Asks questions and generates
data about what is needed to
address the problem or issue
Analysis:
Describes techniques on how to
use the data and results

briefly explained Conduct Needs Assessment and
Analysis
Assessment:
A needs assessment can be conducted in six basic steps:
1- Conduct focus group discussions with community leaders and authorities to
identify key audience, their characteristics and identify problem areas to
generate solutions
2- Identify major topics that relate to the need from focus groups discussions as
well
3- Develop a survey: based on the previous steps
4- Conduct additional focus groups to refine the survey
5- Identify characteristics of learners through knowing about demographic
information, number of siblings, means of income aspirations for the future…
6- Tabulate the results of the survey and interview
Analysis:
After data is collected, and
gaps are identified, the
needs are categorized that
will be translated into
“intended outcomes”.

Start Class activity 1: time limit 10’

briefly explained
II- CONTENT AND
METHODS
State Intended
Outcomes
Develop educational
objectives about what
the learner will be able
to do after
participating in the
curriculum
Select Content
Define what will the
learner need to know:
Knowledge, Skills,
Attitudes
Design Learning Experiences
Develop learning styles and
types of activities with
description as well as
Assessment methods

Class activity 2: time limit 10’

briefly explained
III- IMPLEMENTATION
Produce Curriculum
product
Curriculum product is
produced and includes
suggestions for
evaluation criteria
Test and Revise
Curriculum
Includes suggestions
to select test sites. A
sample evaluation
form is provided
Implement Curriculum
Strategies to promote and use
the curriculum are discussed in
this step

Curriculum Development Phases and Steps briefly
explained
Suggestions for evaluation criteria
The curriculum produced is evaluated to see if it meets minimum required standards:
Does the curriculum product fill a genuine need where no suitable products exist?
Does the curriculum explicitly deal with important knowledge, attitudes, skills, aspirations, or behaviors for the
target group?
Is the subject matter content practical and likely to be useful throughout life?
Does it relate meaningfully to current youth concerns about their immediate and future life?
Are intended outcomes well stated?
Is the curriculum product economical in terms of the time and effort required to achieve learner outcomes?
Does the curriculum provide opportunities for sequential learning?

briefly explained
EVALUATION
Design evaluation strategies
Evaluation is a phase and step. It is
formative and summative
Formative: Applied during the
different curriculum development
phases
Summative: it reports on and
measures the outcomes of the
curriculum
Reporting to stakeholders
Report results to difference concerned
parties, including funding and policy decision
makers

Difference between Curriculum and
Program evaluation
The different definitions given:
1- Both terms used interchangeably
2- Program evaluation is a component of the larger Curriculum evaluation
3- Program evaluation occurs after the curriculum has been adopted and
implemented (used in the context of this presentation)

Program aspects to evaluate
NEEDS
INTENTIONS
ASSESSMENT
ENTRY CHARACTERISTICS
INSTRUCTION
INDIVIDUAL DIFFERENCES
RESOURCES
IMPLEMENTATION
RESULTS

Evaluation points and their place in the Curriculum development model
Needs
Evaluation

Aspects: questions to ask and data sources
NEEDS
1- Is the need that the program was
designed to meet still significant?
2- Is the need recognized by the
learners, teachers and gatekeepers?
3- Is the program based on the
ascertained needs?
4- What other significant need could
the program meet?
Questions to Ask Data Sources
1- Review of needs assessment data
2- Rerun of all or part of needs
assessment
3- Interviews and questionnaires
4- card-sorts with students, parents,
teachers and others
5- Analysis of program results

Intentions
Evaluation

INTENTIONS
1-Are the Curriculum intentions clear to
and adopted by teachers and students?
2- What other intentions are apparently
being pursued?
3- Are teachers and students recognizing
and giving priority to critical objectives?
4- What changes need to be made in the
aim and objectives?
5- What additional objectives should be
included?
1- Classroom observations
2- Analysis of students’ work
3- Projects, folders, tests,
examination responses
4- Analysis of lesson plans, teaching
materials, tests, examinations
5- Interviews with students, teachers
and
parents

Assessment
Evaluation

ASSESSMENT
1- Is achievement of the significant
objectives being assessed?
2- Are the measures used consistent
with the curriculum?
3- Are they valid, reliable, efficient and
fair?
4- Are they intrinsic rather than
artificial?
5- How do students feel about
evaluation?
6- Is there adequate formative
evaluation?
1- Analysis of graded student work,
tests, examinations
2- Discussion with teachers and
parents
3- Interviews with students
especially dropouts and low
achievers
4- Classroom observation

Evaluation points and their place in the Curriculum development model Entry
Characteristics
Evaluation

ENTRY CHARACTERISTICS
1- Do students actually in the program
correspond to those expected?
2- Does the student description give
teachers sufficient information about
students?
3- Are students who lack prerequisites
receiving remediation?
4- Is student difficulty indicating a need for
additional prerequisites?
5- Are pretests being used to provide
teachers with all the critical information
about the students?
1- Students complaints of difficulty,
easiness or inappropriateness of
program
2- Gender or ethnic rations
3- Age distribution
4- Student failure, transfer,
absenteeism and dropout rates
6- Interviews with teachers

Instruction
Evaluation

INSTRUCTION
1- Is the planned content being presented?
2- What changes or additions to content need
to be made?
3- How well are the various teaching strategies
being used?
4- Do the instructional strategies engage and
interest students?
5- Can students make input into the
instructional plan?
6- Is there opportunity for social learning?
7- Is instruction free of gender, ethnic or other
kinds of bias?
8- What new strategies and/or content should be
suggested in the next edition of the curriculum?
2- Analysis of students’ work and test results
3- Student and teacher survey
4- Student absenteeism and dropout rates
5- Suggestion box and telephone answering
machine for students’ suggestions
6- Card-sort of topics by students and teachers
in order of importance and quality of teaching:
list topics and have students say which they
enjoy most, think most important, difficult or
interesting

Individual
differences
Evaluation

INDIVIDUAL DIFFERENCES
1-Are learning problems promptly identified
and remediated?
2- Are faster learners recognized and treated
appropriately?
3- Are aptitude, cultural, and class difference
respected by teachers?
4- Are learners' different ways of learning
recognized and honored?
5- Is the program accessible to students
with special needs?
6- Is the classroom a friendly place for all kinds
of students?
1- Disaggregation of achievement data;
classroom observation; interviews with
students, teachers, parents.

Resources
Evaluation

Resources
1- Are the materials, equipment, and facilities
specified being used?
2- Are students enjoying interacting
with the materials and are better materials or
equipment now available?
3- Are facilities comfortable, attractive, and
appropriate for the instruction?
4- Are the teachers good role models for
students?
5- Are other personnel identified in the
curriculum involved? Are parents involved?
6- Are all students completing the curriculum
on time? Is the time of any student wasted?
1- Site visits and observation: discussion
with teachers, students, administrators.
2- Examination of materials
3- equipment availability and utilization;
teacher complaints of difficulty or lack of time
4- teacher time allocation; requests for in-
service training; teacher qualifications;
student/teacher ratios

Implementation
Evaluation

IMPLEMENTATION
1- Was implementation completed smoothly
and on time?
2- Were approvals received?
3- Was there support from colleagues,
administrators, officials, parents and
community?
4- Were the media supportive?
5- Were teachers willing or enthusiastic?
6- Was training sufficient and well received?
1- Observation and records of implementation
process
2- Analysis of school board and committee
discussions
3- Letters and articles in newspapers
4- Telephone calls to administrators and
officials
5- Evaluation of training sessions

Results
evaluation

RESULTS
1- What was the average level of student
achievement?
2- Did any specific groups of learners
underachieve?
3- Did all students achieve the critical
objectives?
4- What other effects did the program have on
students, teachers, and other people and how
much did students and teachers enjoy and
appreciate the program?
5- How did it change them intellectually,
socially, and personally? Do students now use
vocabulary and concepts taught in the
program?
1- Analysis of test results; student attainment
and achievement; progression rates
2- Interviews with students, teachers, parents,
administrators, employers, and others
3- Analysis of written work by
students and teachers; photographs audio and
Videotape of events and discourse
4- Review of suggestion box, course
evaluations, and student or teacher journals
5- Analysis of enrollment, dropout
rates, absenteeism, disciplinary incidents.

RESULTS (continued)
6- What were the greatest strengths and
weaknesses of the program?
7- How did administrators, parents,
and others respond? Did the results justify
the expenditures of money, time, and
energy?
8- How did the program compare
with previous or alternative programs?
6- Long-term follow-up of students; graduate
employment, earnings and admission to
further study

Recapitulation chart of all the evaluation items
Results
evaluation
Implementation
Evaluation
Resources
Evaluation
Individual
differences
Evaluation
Instruction
evaluation
Entry
Characteristics
Evaluation
Assessment
Evaluation
Intentions
Evaluation
Needs
Evaluation

Class activity 5: time limit 10’

Pogram Evaluation.pptx

Recommended

Recommended

More Related Content

Similar to Pogram Evaluation.pptx

Similar to Pogram Evaluation.pptx (20)

Recently uploaded

Recently uploaded (20)

Pogram Evaluation.pptx