3. Research Evaluation
Produces generalizable
knowledge
Judges merit or worth
Scientific inquiry based on
intellectual curiosity
Provides information for
decision-making on specific
program
Advances broad
knowledge and theory
Conducted within setting of
changing actors, priorities,
resources, & timelines
Controlled setting
Blome (2009)
– Office of Program Analysis and Evaluation, National Institute of General Medical Sciences
4. Research Evaluation
generalizes particularizes
designed to prove something designed to improve something
how it works? how well it works?
what is what is valuable
provides the basis for drawing
conclusions
provides the basis for decision-
making
Mathison (2007)
5. American Evaluation Association
Research Evaluation
Seek to generate new knowledge Information for decision making
Researcher-focused Stakeholder-focused
Hypothesis Key Questions
Methods Analysis
Make research recommendations Recommendations based on key
questions
Publish results Report to stakeholders
6. Evaluation
critical component
determine the value of
the program, its activities,
and staff in order to make
decisions or to take
actions regarding the
future
measure the
delivery of
services
(the process
evaluation) and
outcomes
(product
evaluation)
http://web.stanford.edu/group/design_education/wikiupload/2/27/Helen_Evaluation.pdf
improvement of the guidance program and
gives direction to necessary changes.
7. Importance of Evaluating the Program
Gibson and Mitchell (1995) define program evaluation as a systematic set
of data collection and analysis of activities, undertaken to determine the
value of a program in order to aid management, program planning, staff
development, public accountability and promotion.
Shertzer and Stone view evaluation as necessary to provide for the
effectiveness of achieving program goals, in relation to specific standards.
8. Importance of Evaluating the Program
verifies or rejects practices by indicating what works and what does not,
and shows the extent to which an activity is effective. It helps the
implementors to do away with unproductive innovations.
provides a basis for improvement in terms of operation and
implementation strategies.
suggests a continuous search for better ways of doing things, and a
willingness to look at performance, and increases the search for
improvement.
provides an insight into the program, and helps implementors to
understand their functions and the consequences of what they do.
places responsibility on individuals, and increases the participation of
beneficiaries. It helps in the allocation of roles and responsibilities.
9. determine the impact of the guidance program on students, faculty,
parents, and school climate;
know if they are accomplishing their goals;
identify what remains to be accomplished;
identify effective components of the program;
eliminate or improve less effective components of the program;
adapt and refine the guidance program and implementation process;
identify unintended consequences of the program (both positive and
negative);
identify other areas that need to be addressed;
establish goals for the counselors’ professional development;
determine staffing needs and workload adjustments;
determine additional resources required to adequately carry forward the
program; and
provide accountability information to educators and the community
Importance of Evaluating the Program
10. Evaluation results should be used to make further program
improvements.
Counselors and administrators will use the results to make modifications
to the program
to compare the implemented program with the program standards.
Administrators and policy-makers will utilize the evaluation to make
decisions about the content, quality, and effectiveness of the services and
to allocate financial and staffing resources for the program.
They also will utilize the information to describe the program to the
community or to seek the community’s support for program
improvements.
Importance of Evaluation Results
11. Audiences of the Evaluation
those who have been recipients of the program such as students,
parents, and teachers;
those who have implemented the program such as counselors and
guidance program managers;
those who have administered or set policy such as administrators,
superintendents, school board members; and
those who have supported the program, either financially or
personally such as taxpayers, volunteers,
and community groups.
12. Evaluation Procedures
Identification of goals to be assessed
Such objectives should be clearly stated, concise, specific and measurable. An
example of such an objective would be: make students attend a career fair by
the end of the first term.
Development of an evaluation plan
identify the most appropriate way of judging the extent to which a program
has achieved its goals and objectives. There should be specific information on
how the data is collected, when it is collected, and by whom.
Application of the evaluation plan
This is when data collection and analysis take place.
Utilization of the findings
the findings offer an opportunity to determine future program improvements.
13. Stating the evaluation questions
Determining the audiences/uses for the evaluation
Gathering data to answer the questions
Applying the predetermined standards
Drawing conclusions,
Considering the context
Making recommendations
Acting on the recommendations
Eight Process
Questions to be Answered through Evaluation
How effective have the program improvements been?
Does the program meet the program standards?
Have students become competent in the high priority content areas?
How well are counselors performing their roles?