A program evaluation on the effectiveness of tlar reading intervention using cipp

Program Evaluation
on the Effectiveness of
TLAR Reading Intervention
Using CIPP: Basis for Program Reimplementation, Improvement and
Modification
A proposal presented
To
Dr. Bert Tuga
CIN 504
Curriculum Evaluation
by:
RIVERA, JAESSER G.
2017

1.0 INTRODUCTION
Nowadays, different competencies are emerging to develop holistic learners in
response to the need for globalization and needs to develop 21st century learners. It is
noticeable that various approaches, theories, and technologies emerged already to
support these competencies such as multiculturalism, interdisciplinary approach,
learner-centeredness, focus on the life skills, and integration of technologies and online
learning etc. As the field of education progresses by means of foci it seems that reading
becomes weakened and worst neglected. As cited by Quinn(2015),those students who
can successfully navigate the reading challenges of online learning are having
increased engagement and flexibility, thus earning the credits they need to successfully
graduate (Tapscott, 1998). The statement gives an intensive highlight on the need to
refocus in revitalizing reading abilities/competency.
In the Philippines, English language is one of the most important tools in order to
achieve a certain competency as it is used mainly in the intermediate level as medium
of instruction in the major subjects like Science, Mathematics, and English. Meanwhile,
the Philippines already implemented some interventions regarding strengthening the
skills of learners to better comprehend English language. Obviously, reading ability is
one of the prime skills everyone has to master in order to increase proficiency in
subjects that use English language as medium of instructions.
Recently, the Department of Education funded the ECARP. ECARP or the Every
Child is a Reader Program is a reading intervention funded by the National office just to
vitalize the reading ability of every child. In the recent Department Order Number 18
series of 2017 released by the department secretary, it advocates encouragement to
strengthen the reading skill of Filipino learners. Although not stated on what language
the ECARP focuses still there is a remarkable initiative in the national support system.

The program aims to develop in Filipino children the literacy and
numeracy skills, and attitudes which will contribute to lifelong learning kills or
learners from Kindergarten to Grade 3 following the K to 12 Basic Education
Curriculum by establishing a sustainable and cost effective development
system..
Nonetheless, among the emerging skills and competencies of the 21st century
holistic learners, reading skills appears so weak which is directly proportional to
comprehension and deeper understanding of a certain ideas particularly ideas in
subjects which medium of instruction is English. In order to address the gap and
weaknesses various reading intervention programs are now available to market and
online industry especially these days that almost everything show up in just a click of a
hand.
Inspired by the ECAR program, Tibig Elementary School-a non-sectarian public
elementary institution launched its contextualized version of the intervention. The
program was called TLAR (Tibig Learners are Readers). The reading intervention was
launched in an aim to strengthen the reading proficiency of Tibig Elementary School
learners, which were severely ailing based on the results of the PHIL-IRRI pre-test
examinations on the third graders.
The prime reasoning on the deviance of TLAR from the ECARP is the movement
of contextualization in the curriculum. Contextualization is an emerging current trend in
the Philippine education. In an article published by Jenny Lee Utech(2008) she noted
that in order to contextualize curriculum, teachers use authentic materials, activities,
interests, issues and needs from learners’ lives to develop classroom instruction so that
learners learn language by teaching the skills using the authentic contexts in which
students must use those skills in the real world.
In one hand, one of the main difficulties a student may face as a result of being
unable to read properly is a knock to their confidence. Trying to keep up with the

mainstream group but being unable to do this can cause serious emotional strain; it can
make you feel inferior, totally useless, and as if you're not as 'smart' as the rest of your
class. To correct this, one of the biggest aims of reading intervention is to restore the
confidence of the affected person. This takes time - and lots of effort - but can be done.
It demands patience from the tutor, as well as the ability to teach the student at their
own pace - however slow this may be. Everyone learns at different speeds, and the
beauty of acquiring knowledge slowly is that it's more likely to be retained.
Intervention is also a valuable method of increasing children's reading speed,
which can benefit them by teaching them to read faster, whilst still retaining large
chunks of important information. Also, by improving their reading fluency, students gain
the ability to read smoothly and more efficiently which can be an advantage to all parts
of their education. Most reading intervention programs also help to build and expand a
student's vocabulary. Most intervention programs are focused on the individual, and the
exact areas where they are experiencing trouble. The role of the reading tutor is to
analyze the specific areas where concerns are arising, and from here, look at the best
ways of educating the student on how to overcome these. A timetable or planning sheet
is often used to plan when and how various areas can be tackled, and how they interlink
( http://www.teachnology.com/tutorials/teaching/readingintervention.html).
According to Torgesen (2005), Effective interventions require skillful teaching.
The most effective intervention teachers are likely to be those with the most training and
experience. However, in the absence of well-trained and experienced intervention
specialists, less experienced teachers, or even qualified paraprofessionals, can deliver
effective interventions if they are trained to use a well-developed, explicit, and
systematic intervention program. Many of these programs are available, and they
provide a useful “scaffold” to help less experienced teachers provide powerful
instruction. A good rule of thumb is that the less experienced the teacher, the more
structured and “scripted” the intervention program should be.

On the other hand, after the completion in the implementation of any program is
the need to determine the effectiveness of such so that at any time there is a need to
employ program concerning the same issue/ problem/ conflict for the program will be
basis for reimplementation and/or modification.
Evaluation was conceptualized by Ralph Tyler (1991) as a process
essential to curriculum development. The purpose of evaluation was
stated as to determine the extent to which the curriculum had achieved its
stated goals(Tunc, 2010).
Program evaluation is a valuable tool for program managers who are
seeking to strengthen the quality of their programs and improve outcomes for the
children and youth they serve. Program evaluation answers basic questions
about a program’s effectiveness, and evaluation data can be used to improve
program services. In this brief, we define program evaluation, address common
concerns program managers and practitioners have regarding evaluation, and
outline five major reasons why conducting a program evaluation can benefit an
out-of-school time program (Metz, 2007).
According to Tunc (2010),it is a fact that evaluation may be conducted for a wide
range of reasons in every part of our life. In terms of education, it can be stated that the
main purpose of evaluation is to obtain information about student and teacher
performance along with classroom interactions. In the same way, the aims might also
include to identify strengths and weaknesses of particular activities in a program. Given
the fact that there is a vast amount of evaluation models and processes available in the
reservoir of educational resources, it is somehow difficult to choose an appropriate
model. Even so, the need to evaluate a certain program is vital need for improvement
and modification for future enhanced use.

The primary purpose of this paper is to intensively evaluate a reading program
employed in the context of Tibig Elementary School. Using CIPP evaluation model
developed by Stufflebeam this paper will serve as a basis for program
reimplementation, improvement and modification.
The CIPP Evaluation Model was originally developed as a means to
systematically provide timely evaluative information for use in decision making. Use of
the CIPP Model thus is intended to facilitate educational improvement through a
proactive approach to evaluation (Stufflebeam, 1971).
Stufflebeam’s Context, Input, Process, and Product evaluation model is “a
comprehensive framework for conducting formative and summative evaluations of
projects, personnel, products, organizations, and evaluation systems” (Stufflebeam &
Shinkfeld, 2007). The model originated in the late 1960s to provide greater
accountability for the U.S. inner-city school district reform project. It was to address the
limitations of traditional evaluation approaches (Stufflebeam, 1971). The CIPP
evaluation model “is configured especially to enable and guide comprehensive,
systematic examination of social and educational projects that occur in the dynamic,
septic conditions of the real world . . .” (Stufflebeam & Shinkfeld, 2007, p. 351) as cited
in the paper of Zhang, et. Al (2011) .
In this paper, it is important to note that a contextualized reading intervention
program also needs to be evaluated using an appropriate evaluation model and since
the program has already implemented the tendency is to use CIPP model not because it
is the most used or abused neither it is the most popular but because it is the most
appropriate.

Various studies have been focusing on the intervention in the reading process
both locally and globally. In as much as interventions are focused some studies are
mentioning the need to evaluate a certain program to see the effectiveness and
efficiency. With that, several researches involving reading intervention and program
evaluation are gathered.
In 2013, Nora Verdan and Mary Ann Tarusan of the University of Mindanao
conducted a study on the reading ability of Grade Six students in Davao and parent’s
involvement in their education. The study attempted to relate the role of parental
involvement as an additional factor possibly influencing ability. Their study reveals that
parent involvement may not be the most influential factor in improving academic
outcomes. Other factors such as teacher quality, reading materials, and student’s lack
of interest in reading and alignment of curriculum may likely impact student performance
more. In relation to this paper, in could be noticeable that reading really play a vital role
in education and an intervention like parental involvement is an additional factor that
can be considered.
Figuerres (2011) assessed the Reading Literacy Extension Program of the
University of Northern Philippines in Vigan City through determination and analysis of
reading competencies of during the implementation of the program and assess the
program as a whole. Findings showed that the RLEP improved remarkably the reading
competencies of pupils and strengthened their reading skills. The study also revealed
that the use of mother tongue language is proposed to better reading comprehension.
Hayne and Ghosh (2008) applied a system-oriented evaluation
methodology to the Strategic Collaboration Model a management model using CIPP.
There were four types of evaluation in the study. C- covered Mentoring as primary
developmental process in succession planning. I- Determined the sources directed at
the program. P- Examined program evaluation. P- Examined program outcomes in a
mentoring-centered succession management. The study revealed that renewal; and

transition of leadership in organization is necessary and vital to maintain competitive
advantage.
Tunc aimed to evaluate the effectiveness of Ankara University Preparatory
School program through the perspectives of instructors and students. To this end, the
CIPP (context, input, process, and product) evaluation model developed by Stufflebeam
(1971) was utilized. The results were displayed under four parts of the CIPP (context,
input, process, and product) evaluation model developed by Stufflebeam (1971). Under
Context the researcher used survey, document review, and interviews. For Input and
Process, he used questionnaires and interviews with the instructors. Lastly, he
qualitative and quantitative analyzed questionnaires and interviews. It was found out
that the program needed some revisions to make better use of existing opportunities.
Rodgers (2016), conducted a CIPP program evaluation of the implementation of
a Response to Intervention (RTI) Program in a rural school district. The focus of this
study was an RTI program in its second year of full implementation in kindergarten and
first grade in 15 elementary schools. This mixed-method study utilized data gathered
from reading achievement and special education referral data, district-level and school-
level administrator interviews, a teacher survey, and focus groups. Findings from this
study indicated there was no significant difference in reading achievement and special
education referral data in the 2 years of program implementation.
Quinn (2015) evaluated an program based on the impact of “Read to Learn”
Model on Alternative High School Students’ LEXILE Levels and Reading Achievements.
The study is a program evaluation of the effectiveness of a reading initiative in an
alternative school utilizing the Stufflebeam CIPP evaluation model. To adequately
gauge the impact of the “read to learn” program, qualitative and quantitative evidences
were gathered. Qualitative evidence was gathered from multiple interviews from
building-level staff as well as district office staff responsible for the design and
implementation of the model. Common themes that were determined by multiple
mentions during the interviews were analyzed from the interview responses to evaluate

the context, input, and process of the model. Quantitative data were analyzed in the
product component of the program evaluation. These data were generated from
nationally normed reading assessments with students who participated in reading as the
primary mode of learning in all content areas The evaluation found the “read to learn”
model to have been properly implemented according to the CIPP model assessment.
The study also found the program had been implemented with fidelity and consistency.
Though no significant reading impact was evidenced, the “read to learn” model was
perceived to have been successful.
Jarvis (2016) evaluate the Direct Instruction programs, Reading Mastery and
Corrective Reading, from SRA McGraw-Hill Publishing Company, which were being
used as a school-wide reading intervention. The CIPP (Context, Input, Process,
Product) Model was used as the evaluation tool for this program evaluation. Research
questions were related to the Context of the problem for which the program was
addressing, the Input or resources available to address the problem and any strategies
that had been tried, the Process or implementation of the program, and the Product or
outcomes of the program. The achievement data and survey data from this evaluation
indicate that some students saw gains in their reading scores and most of the students
believed SRA was beneficial to them.
Buie and Simeon in 2014 determined whether or not the Reading Plus program
impacted student academic achievement in reading for those students enrolled in the
program in grades four and five based on the student Lexile scores generated from the
Scholastic Reading Inventory (SRI). A full evaluation was conducted using Daniel
Stufflebeam’s Context, Input, Process, Product (CIPP) model. The Reading Plus
program evaluation using the CIPP model consisted of three steps initially theorized by
Stufflebeam and focused on the product of the targeted program. The first step was
delineating, which involved assessment of the Reading Plus program based on program
expectations by administrators. The second step in the evaluation process was
obtaining, which resulted when product information was obtained through both interim
and final measures of data from those students who were enrolled in the Reading Plus

program and by analyzing responses of teachers to survey. The third step in the
evaluation process was providing. Varying degrees of information and data from the
Reading Plus program evaluation were provided to decision makers Reading Plus was
found to be an effective intervention program and cost effective for the school which
was evaluated.
This paper is premised on the conceptual framework inspired by CIPP model and
SWOT analysis.
C I P P
S W O T
Decision
Reimplementati
on
Improvement Modification
TLAR

With the information and previous studies, the researcher would like to determine
the effectiveness, loopholes and points for modification in terms of context, input,
process, products. The researcher also expects different revealing factor in the reading
intervention away from the previous studies conducted using the CIPP. With the pluses
of the model to be used in the program evaluation the revealed points will serve as
basis for improvement for future use; both in the internal merits and external merits of
the program. The result of this paper will also reveal points to modify for future
reimplementation in various local and global contexts with similar circumstance and
condition.
In order to understand this paper to the fullest of its intention, concepts and
technical terminologies are presented;
CIPP Model- an evaluation model originally developed by Daniel Stufflebeam to provide
timely information in a systematic way for decision making,which is a proactive
application of evaluation.
Intervention- pedagogical technique that seeks to reform an older system or practice.
LEXILE- used to monitor a reader’s growth in reading ability over time.
Program Evaluation- efficiency, effectiveness, and accountability of a department,
program or agency. The purpose is to determine whether the program is efficient
in terms of using resources wisely to perform the needed.
TLAR- A contextualized reading program in a local elementary school inspired by
ECARP

2.0 METHODOLOGY
This paper will utilize a descriptive-evaluative design of research and is highly
qualitative in nature but with the touch of quantitative as it is the most appropriate
approaches for this study. These designs will enable the researcher to assess the
effectiveness of the program in details both in quality and of support by quantity.
Random sampling of various stakeholders will be used to minimize biases and
prejudices. In addition to that, in order to get a valid and precise result of the study, this
paper will use triangulation in its instruments. First, qualitative-analysis using CIPP
model. This will delimit the researcher in the vast investigation and maximize description
in every aspect and details in context, input, process, and product. Second, a focused
group discussion among the participants and stakeholders will be used as instrument to
determine the role, function, mode of procedures, and effectiveness in each
perspective. With that, a detailed analysis can be made with high accuracy. Third,
LEXILE measurement of the reading ability of the respondents will be maximized to
gather in quantity the effectiveness of the product. Of all three instruments, a systematic
classification process of coding will be used in the description.
The CIPP Evaluation Model was originally developed as a means to
systematically provide timely evaluative information for use in decision making. Use of
the CIPP Model thus is intended to facilitate educational improvement through a
proactive approach to evaluation (Stufflebeam, 1971).
Model of Stufflebeam,1971 as portrayed in the framework
C I P P

CIPP model will guide this paper in the analysis of context, input, process and
product. With the purpose of analysing the effectiveness, SWOT analysis will be
maximized in each aspect of the model. However each aspect will be guided by the
description of Stufflebeam in his 1971paper and this paper will be guided by the
following questions:
Context:
Context evaluation provides information about the strengths and weaknesses of a total
system to assist in planning improvement-oriented objectives at each level of the
system. Data should also be obtained and filed in retrievable form relative to
opportunities that might be used to achieve institution objectives .Such data would
center on, but not be limited to, funding opportunities to support institution programs .It
would also seem evidential too obtain and record information about the nature and
effectiveness of relevant innovative efforts in other institutions (Stufflebeam,
1971).Specifically this paper will seek these questions:
1. What is the rationale of the program?
2. What is the reason behind the use of such program?
3. What research is the use of this program inclined to?
4. What content needs to be address in the future?
Input:
Input evaluation provides information about the strengths and weaknesses of alternative
strategies which might be chosen and structured for the achievement of given
objectives. This has its purpose to identify and assess alternative program strategies for
achieving given objectives and to provide information to assist in detailing particular
strategies .To fulfil these purposes, an input evaluation unit must possess personnel,
resources, and procedures to be used in conducting ad hoc input evaluation studies
after a decision which specifies new objectives. Then it is necessary to inquire how the

chosen objectives can be efficiently and effectively achieved. Further, there should be
an analysis of the strengths and ease of use of each strategy relative to achievement of
the given objectives. If a strategy aids achievement of one objective, but hinders
another, the relative effect of the strategy on the overall program should be analysed
(Stufflebeam, 1971).Specifically this paper will seek these questions:
1. How did the stakeholders execute their roles?
2. What are the perceptions of the participants and stakeholders toward the
program?
3. How do the stakeholders prepare in the implementation?
Process:
Process evaluation provides information about the strengths and weaknesses of a
chosen strategy under conditions of actual implementation, so that either the strategy or
its implementation might be strengthened. Process evaluation is designed to provide
information during theimplementation stages of a project or program,' which can assist
program managers to operate the program according to its design, improve the program
design as effects are indicated under operating conditions, and to make structuring
decisions which could not be made during the preparation of the program design. A
secondary purpose of process evaluation is to provide a complete description of the
actual program activities. Such a description should be prepared to assist program
replication and to assist in determining why program objectives were or were not
achieved . Techniques which can prove useful for monitoring include daily logs,
observation, interviews, questionnaires, open-end reaction forms, and se, forth. A
complete file of process data should be maintained(Stufflebeam, 1971).Specifically this
paper will seek these questions:
1. What are the commonly encountered problems in the implementation?
2. Was the program implemented with consistency?

3. How was the program executed?
4. Were the resources available in the implementation?
5. Which part of the implementation is the most difficult?
6. Up to what extent the program was used?
Product:
Product evaluation provides information for determining whether objectives are being
achieved and whether the change procedure which has been employed to achieve them
should be continued, modified, or terminated. The purposes of product evaluation are to
relate outcomes to objectives and to assess the overall worth of a procedure in terms of
its effects. Variables for product assessment should be delineated in terms of the
objectives which have been selected and in terms of the overall problems that a prefect
or program has been designed to solve. The product assessment person and the
program personnel should define criterion variables which relate directly to objectives.
Product information should be obtained by taking both interim and final measures of
product criterion variables. To the extent possible, such measures should be obtained
so that product and context data can be compared. In determining the extent to which
objectives were achieved, one should consider the effect of the product on the overall
needs or opportunities which motivated the development of the objectives .Major
approaches to product evaluation use true experimental design, quasi-experimental
design, and comparison of products achieved with specified standards . (Stufflebeam,
1971).Specifically this paper will seek these questions:
1. How well the respondents improve in reading abilities?
2. Is there a significant difference in terms of skills in the pre and post
implementation?
3. Did the program affect the stakeholders in positive or negative way?
4. Into what degree does the program meet the expectation in the results?
5. How recommended the program is for reimplementation?

6. What points in the results must be refocused in the next implementation?
7. Which are the weaknesses of the results?
The quantitative data will use SPSS 15.0 to treat the data in the LEXILE
measurement.
The researcher will provide the concerned authority an assurance that all of the
data obtained from the content of reading intervention will be kept anonymous and no
personal issues arising from this research will be brought against the aforementioned.
The reading intervention (TLAR) to be used will only be analysed and evaluated using
CIPP Model and the researcher will not critique the designer of the reading intervention
and/or any of its stakeholders.
The researcher will also apply appropriate and adequate citations in writing the
paper. The composed output of the researcher will not violate any intellectual property
rights and shall not commit plagiarism. Furthermore, the researcher will give proper
acknowledgement to the authors of the used related literature and related studies.

References
Buie, D., County, J., Calcutt, R., & Simeon, E. (2014). Program evaluation of "reading
plus": Study of the impact on reading achievement at the school level in moore
county schools (Order No. 3682144). Available from ProQuest Central; ProQuest
Dissertations & Theses Global. (1657424657). Retrieved from
https://search.proquest.com/docview/1657424657?accountid=173015
Figuerres, Ocarna (2011). The Reading Literacy Extension Program of the University of
Northern Philippines. UNP Research Journal. Vol. XX. Retrieved at:
https://go.galegroup.com/ps
Haynes, Ray and Ghosh Rajashi (2008). Mentoring and Succession Management: An
Evaluative Approach to the Startegic Collaborative Model. Retrieved at:
https://search.proquest.com/openview/2ad6faec35218b8d12e2ed1f5c493bf9/1?p
q-origsite=gscholar&cbl=36534
Jarvis, Nita M (2016). Gardner-Webb University, ProQuest Dissertations Publishing
Retrieved at:
https://search.proquest.com/openview/4dd918ebb0caeac139cbdf339de0e31b/1?
pq-origsite=gscholar&cbl=18750&diss=y
Metz, Allison J. (2007). Why Conduct A Program Evaluation? Five Reasons Why
Evaluation Can Help An Out-Of-School Time Program. Washington DC: The
Atlantic Philanthropies retrieved from:
https://cyfar.org/sites/default/files/Child_Trends-07_10_01_RB_WhyProgEval.pdf

Quinn, James (2015). A Program Evaluation of the Impact of a "Read to Learn" Model
on Alternative High School Students' Lexile Levels and Reading
Achievements. Theses. Accessed : September 7, 2017 Available at :
http://digitalcommons.gardner-
webb.edu/cgi/viewcontent.cgi?article=1000&context=education_etd
Rodgers, A. G. (2016). Response to intervention: A program evaluation of
implementation in a rural school district (Order No. 10191182). Available from
ProQuest Dissertations & Theses Global. (1853453608). Retrieved from
https://search.proquest.com/docview/1853453608?accountid=173015
Stufflebeam, Daniel(1971). The Relevance of the CIPP Evaluation Model for
Educational Accountability.Document Resume. New Jersey: The Ohio State
University retrieved September 9, 2017 from
http://files.eric.ed.gov/fulltext/ED062385.pdf
Torgesen, Joseph (2005). A Principal’s Guide to Intensive Reading Interventions for
Struggling Readers in Reading First Schools. Accesed: September 11, 2017
Available at:
https://www2.ed.gov/programs/readingfirst/support/principal.pdf
Tunc, Ferda (2010). Evaluation Of An English Language Teaching Program At A Public
University Using CIPP Model. Thesis. Retrieved at :
https://etd.lib.metu.edu.tr/upload/12611570/index.pdf
Utech, Jenny Lee (2008). Contextualized Curriculum for Workplace Education: An
Introductory Guide. Massachusetts Department of Elementary and Secondary
Education Adult and Communit Learning Services. Retrieved from:
https://www.umass.edu/roundtable/projects/Integrated%20curr_guide%20p1.pdf

Verdan, Nora and Tarusan, Mary Ann (2013). Parental Involvement in the Education of
Grade Six Pupils and their Reading Ability. Thesis. Retrieved at:
http://dx.doi.org10.7718/iamure.v6i1.658
Zhang, Guilli, et. Al, (2011). Using the Context, Input, Process, and Product Evaluation
Model (CIPP) as a Comprehensive Framework to Guide the Planning,
Implementation, and Assessment of Service- learning Programs. Thesis.
Accessed: Sep 7, 2017.Retrieved at
https://www.researchgate.net/publication/262918987_Using_the_Context_Input_
Process_and_Product_Evaluation_Model_CIPP_as_a_Comprehensive_Framew
ork_to_Guide_the_Planning_I mplementation_and_Assessment_of_Service-
learning_Programs
Online Sources
https://doe.sd.gov/octe/documents/WhatDoestheLexileMeasureMean.pdf
https://www.merriam-webster.com/dictionary
https://www.learningtogive.org/resources/program-evaluation
https://www.igi-global.com/dictionary/educational-intervention/33094

A program evaluation on the effectiveness of tlar reading intervention using cipp

More Related Content

What's hot

Similar to A program evaluation on the effectiveness of tlar reading intervention using cipp

More from Department of Education-Lipa/ Philippine Normal University

Recently uploaded

A program evaluation on the effectiveness of tlar reading intervention using cipp