Introduction• Federally mandated public after-school tutoringis not always reaching the children its intendedto help, and when it does, it does not alwayshelp as much as it could (Toppo, 2008).• Questions are being raised about theeffectiveness of tutoring for underachievingstudents, the accountability of tutors, andacademic return on this investment (Buczynski,2008).
Introduction (con.)• Theres still a dearth of research evidence toshow whether one of the federal measuresleast-tested innovations--a provision that callsfor underperforming schools to provide after-school tutoring--has an impact on studentachievement (Evidence Thin on Student Gains,2007).
Background of the Problem• Not surprisingly, 40% of all urban districts are requiredto offer supplemental services (Ascher, 2006).• Gewertz (2005) says that states must evaluate theeffectiveness of free tutoring under the federal NoChild Left Behind Act, but a lack of resources mightforce them to compromise on the rigor of thoseevaluations.• By anecdotal accounts, most states are not far along indesigning evaluations and many experts question theircapacity to design and implement high-qualityevaluations (Gewertz, 2005).
Statement of the Problem• Saulny (2006) asserts that even for thosestudents who are getting tutored, there has yetto be a scientific national study judgingwhether students in failing schools are receivingany academic benefit, and there is noconsensus on how that progress should bejudged.
Research Question # 1What is the relationship betweentutoring effectiveness and studentachievement based on anelementary school’s rating on theCharacteristics of EffectiveTutoring Scale and its TAKSCumulative Met Standard inreading?
Research Question #2What do administrators reportabout the effectiveness ofimplementing Title I tutoringprograms in elementary schoolsas indicated by their rating onthe Characteristics of EffectiveTutoring Scale?
Research Question #3What do elementary schoolteachers report about theeffectiveness of implementingTitle I tutoring programs inelementary schools as indicatedby their rating on theCharacteristics of EffectiveTutoring Scale?
Research Question #4Is there a difference in theratings of administrators andteachers on the Characteristicsof Effective Tutoring Scale?
Null Hypotheses• H1 - There is no relationship between tutoringeffectiveness and student achievement basedon an elementary school’s rating on theCharacteristics of Effective Tutoring Scale andits TAKS Cumulative Met Standard in reading.• H2- There is no difference in the ratings ofadministrators and teachers in their overallscores on the Characteristics of EffectiveTutoring Scale.
Purpose of the StudyThe purpose of the study will be to investigate thestructure of tutoring programs currently employed bypublic schools. Faculty members will rate theeffectiveness of the campus tutoring programs bycompleting the Characteristics of Effective TutoringScale. Descriptive data will be included in the study toindicate factors that may be associated with theimplementation and evaluation of Title I tutoring andfurther analysis will determine if any significantrelationship exists between the quality of a Title Icampus tutoring program and its TAKS Cumulative MetStandard. The study will also investigate whetheradministrators and teachers differ in rating theircampus tutoring program.
Significance of the Study• Results of the study will provide an overview of theeffectiveness of the tutoring methods that are utilized.• Implications for school administrators extractedthrough the study may substantiate the currentpractices that are in place.• On the other hand, the study may serve as an apparatusof transformational change in streamlining the contentand structure of tutoring programs.• Ensuring that compensatory education programs areeffective will enable educators to advance a stepfurther in the quest to ameliorate the achievement gapbetween at-risk students and students of mainstreamAmerica.
Conceptual FrameworkWhat is “tutoring effectiveness?”According to Gordon, Morgan, Ponticell, and O’Malley (2004):• From what researchers now know, there seems to be at least 10components that support high-quality tutoring:• 1. More professionally prepared tutors consistently producesignificantly higher levels of student achievement than tutors withlittle or no special preparation.• 2. Tutors need to use a diagnostic/developmental template toorganize and implement each students tutoring program.• 3. The tutors must be able to track the session-to-session progressof each student in order to modify tutoring content and usestudent academic strengths to overcome weaknesses.• 4. Principles of learning drawn from both cognitive andconstructivist thinking seem to offer the strongest contemporarytutoring methods.• 5. Tutors need to use continuous feedback to help studentsdevelop positive self-images as learners.
Conceptual FrameworkWhat is “tutoring effectiveness?”• 6. Formal/informal assessment needs to be used throughout thetutoring process.• 7. Mentoring/coaching students on learning how to learn throughproviding guidance on study habits, test taking, attention toschool, and learning in general is a significant informal part ofeffective tutoring.• 8. Mentoring/coaching each students parents on sustaining theday-to-day learning process in the home after the tutoring ceasesis an important role for effective tutors.• 9. To facilitate the coaching of parents, it is desirable to conductthe tutoring in the students own home outside of school hours. Ifthis is not possible, a community center or library can be used, butan effort to provide mentoring to the parents should still be made.• 10. Throughout the tutoring, tutors must collaborate closely witheach students classroom teacher. The final measure of theeffectiveness of the tutoring is the short-term and long-termimprovement of the students day-to-day classroom achievement.Close tutor-teacher collaboration will help maximize effectivetutoring.
Review of theReview of theLiteratureLiterature
The Evolution of Tutoring• The tutor-student model is patterned after therelationship that existed between the studentand sage during the age of Socrates (McDonald,2004).• Gordon (1990) contends that tutorial philosophyand methods were forgotten or quietlyabsorbed by tax-supported public schools.
The Achievement Gap• While Americans are mixed on the urgent need toaddress the racial achievement gap, the schoolsresponsibility for the gap, and the practice ofdisaggregation, the Bush administration and NCLBproponents have aggressively wielded the lawsemphasis on achievement gaps as a political tool (Hess,2006).• English (2003) asserts that current methods ineducation have not solved the major sociopolitical andinstructional problems of race or class, and they havenot reduced the gap between the socioeconomic havesand have-nots.
Compensatory Education: Title I• Title I, the best known of the Act’s six titleswas included for the purpose of meeting thespecial educational needs of children of low-income families (Kritsonis, 2002).• For decades, people have argued aboutwhether Title I in itself can be praised forraising test scores among broad groups insociety, or whether it should be condemned fornot closing the gap between poor and richchildren (Borman, Stringfield, & Slavin, 2001).
21st Century Educational Reform: No ChildLeft Behind• As a result of No Child Left Behind (NCLB),substantial new federal spending is allocatedfor supplemental education services, whichmost often mean tutoring (Buczynski, 2008).• Underlying supplemental services is theassumption that academic instruction providedoutside the regular school day by public andprivate organizations will be able to do whatschools cannot - raise the achievement ofstudents in consistently poorly performingschools (Sunderman, 2006).
Shortcomings of Tutoring Programs• In the 299 districts surveyed by the Center onEducation Policy (CEP) in 2005-06, 20% ofeligible students took advantage ofsupplemental services; however, extremely lowpercentages of eligible students enrolling insupplemental services have been reported inHouston (3%) and Philadelphia (5%) (Ascher,2006).
Shortcomings of Tutoring ProgramsStudents could get better access to federallyfunded tutoring programs if lawmakersstreamlined the sign-up process, gave statesand districts money to monitor and evaluatethose services, and took steps to make suredistricts actually spent money for tutoring ontutoring (Borja, 2007).
Shortcomings of Tutoring ProgramsIn a study conducted by the U.S. Department ofEducation from 2003-2005, Anderson and LaGuarda(2005) found impediments regarding the state of freetutoring programs:• Some providers were unable to describe any strategyfor aligning their services with state standards.• Student attendance at after school tutoring was achallenge.• Provider communication with parents and teachers wasseldom very effective.
Shortcomings of Tutoring Programs• Similar findings were observed by Ascher (2006):Although NCLB mandates "highly qualified" teachers forevery classroom during the school day, the law is silentabout qualifications for tutors. Most tutors arecertified teachers, but some are college graduateswithout teaching experience, and 7% are high schoolstudents. Some, but not all, providers prepare theirtutors to work with their instructional programs -preparation ranges from four to 20 hours. Some, butnot all, providers evaluate their tutors.
The Need for Title I Tutoring ProgramEvaluation• The lack of tutorial program supervision by public orprivate regulatory agencies resulted in some tutorsmaking unfair claims regarding academic improvementthat unduly raised student and parent expectations(Gordon, 1990).• As tutoring programs of various types receive attentionas possible solutions to modern educational problems, itbecomes necessary to evaluate them in terms of theirbenefits to the students involved (Von Harrison &Guymon, 1980).• Few research studies, for example, include a controlgroup of students, prohibiting clear conclusions on theeffects of the tutoring; therefore, the long-term effectsof reading interventions need to be investigated(Senesac & Silberglitt, 2008).
Investigating Tutoring Effectiveness• The demand for proven results, extensive evaluations, anddata-driven decision-making has moved the role of thesuperintendent from the sideline to the frontline ofsupporting student achievement (Peterson & Young, 2004).• As great as the need is for similar quantitative studies fromother districts, including those that follow students overmore than one year, there is also need for programobservations that facilitate an understanding of howsupplemental services classrooms are over time and howchildren experience tutoring (Ascher, 2006).• It is imperative that resources be allocated to design andimplement sophisticated evaluations of tutoring efforts(Pearson, 2000).• Tutoring programs must be evaluated rigorously andsystematically in order to determine: which produce thestrongest and most reliable effects on student learning,which produce negligible effects, and which produce no oreven negative effects (Slavin & Calderaon, 2000).
Characteristics of Effective TutoringPrograms• Effective programs require adequate trainingfor tutors, whether these are college students,community volunteers, or other children;second, supervision of tutors is essential(Pearson, 2000).
Garner et al. (2002) provides a framework for effectivetutoring:• Organizational support is one of the most importantaspects to a successful after-school tutoring program.• Providing adequate space to conduct the tutoringprogram and appropriate materials such as books andwriting supplies to implement the program, andproviding a designated person or people to be in chargeof implementing the program are key to programsuccess.• In all of the effective after-school programs reviewed,training of the tutors was a consistent factor.• An after-school tutoring program needs to haveappropriate materials to implement a high-qualityprogram.• Incentives are important to keep tutors and tuteesinvolved in the program.
Summary of the Literature• In conclusion, the literature suggests that tutorsmust be equipped with ample training,resources, and evaluative feedback in order toproduce an effective impact on studentachievement.• Education reform measures promoting research-based programs should encompass tutoring sothat this important form of education canbecome a more potent resource in improvingstudent performance (Gordon et al., 2004).
Research Methods• Current conditions of tutoring programs will bedescribed and analyzed for relationships betweentutoring effectiveness and academic achievement.• For the purposes of establishing a relationship betweenvariables, no logical causal ordering can be implied;student achievement is the criterion variable whilelevel of tutoring effectiveness is the predictor variable.• In addition, two independent variables, the ratingscores indicating the perceptions of administrators andstaff members will also be compared through a t-test.
Subjects of the Study• Houston, Texas has a low enrollment of eligiblestudents in Title I tutoring (3%) according toAscher (2006).• For this reason, it is the interest of the study todiscover how teachers and administrators in theHouston area regard Title I tutoring programsbased upon the Characteristics of EffectiveTutoring.
Subjects of the Study• The study will be conducted in Harris County,the most populous county in the state of Texas.• Harris County includes 23 school districts;nonrandom, purposive and conveniencesampling methods will be used to invite 2school districts to participate.• The two districts invited to participate in thestudy will be purposely selected based on thefact that a large number of their schools areconsidered Title I.
Subjects of the Study• Schools will be invited through clustersampling.• Approximately 45 elementary schools reside inDistricts I and II combined.• Ten schools will be selected from the schooldistricts (n=10) by placing all 45 elementarycampus names in a hat to retrieve the desiredsample size.
Subjects of the Study• Once ten schools have been selected, teachersand administrators will be invited to participatein the study via email.• Convenience sampling will be used in selectingadministrators and teachers.• At least two administrators and 20 teachers percampus is the desired sample size (n=220).
Instrumentation• Thirty statements regarding program administration,program design, family involvement, and tutoringsessions are included in the survey.• Together, these four components of the survey will becombined to elicit an overall score that will determinewhether a school’s tutoring program is deemed ashighly effective, efficient, emergent, or in need ofimprovement.• Information about the reliability and validity of theCharacteristics of Effective Tutoring Scale as a websurvey will be addressed in a pilot study.
Instrumentation• Cooperation in completing the survey is likely due to itsbrevity, readability of responses and the use of Likert-responses; furthermore, “if a survey request isperceived as interesting and easy, the likelihood ofobtaining cooperation will increase” (Biemer & Lyberg,2003, p. 107).• Confidentiality of respondent’s identity and letters sentto participants to explain the significance ofparticipating in the study may increase the responserate.• In completing electronic surveys, “confidentialitymust be assured and guarantees must be provided thatinstallment of the communication package will not leadto virus attacks” (Biemer & Lyberg, 2003, p.201).
Instrumentation• Additional benefits of web survey designs suchas Survey Monkey include “controlled routingand embedded edits” that may “decreasemeasurement error and item nonresponse”(Biemer & Lyberg, 2003, p.201).
Pilot Study• In its original form, Characteristics of Effective Tutoringis a checklist of 30 standards that effective tutoringprograms may have (see Appendix C).• Likert-type responses will be added to the checklist andwill be converted into a web survey on a secure siteusing Survey Monkey in order to elicit responses fromadministrators and staff members regarding thetutoring practices of each campus.• A pilot study will be conducted to ensure thatconverting the checklist by adding Likert-typeresponses yields a reliable and valid survey.
Pilot Study• Through convenience sampling, five teachersand one administrator will be invited tocomplete the survey.• Respondents’ answers as well as any commentsregarding the flow, readability, and relevancyof the questions will be analyzed in the pilotstudy.• Appropriateness of the scale will be evaluatedfor the purposes of the study.• Attention to the layout of the questionnaire willalso be addressed to ensure that the instrumentis clear to respondents.
Procedures• Respondents will be introduced to the Characteristics ofEffective Tutoring Scale through a link provided in anemail.• All submissions will be completed online via the WorldWide Web.• Respondents will choose the location in which tocomplete the web survey.• Identifying information will not be obtained.• Staff members and administrators will respond to eachitem on the survey by indicating the extent to whicheach statement is representative of the campus.
Research Question#1Hypothesis Criterion Variable Predictor Variable StatisticalMeasurementWhat is therelationshipbetweentutoringeffectivenessand studentachievementbased on anelementaryschool’s ratingon theCharacteristicsof EffectiveTutoring Scaleand its TAKSCumulative MetStandard inreading?H1 - There is norelationshipbetweentutoringeffectivenessand studentachievementbased on anelementaryschool’s ratingon theCharacteristicsof EffectiveTutoring Scaleand its TAKSCumulative MetStandard inreading.StudentachievementLevel oftutoringeffectivenessPearson’s rcoefficient ofcorrelation
CORRELATIONAL STATISTICS• The first hypothesis, H1, involves correlational research. Foreach of the 10 campuses included in the study, the first variable,TAKS Cumulative Met Standard value, will be collected via AEISreports.• TAKS Cumulative Met Standard values for will be listed in a SPSSspreadsheet.• Using the values emanated from the descriptive research portionof the study, the arithmetic mean of each school’s rating scale willbe listed in an adjacent column in SPSS.• The Pearson r correlation coefficient will be calculated todetermine whether a significant relationship exists between acampus average score on the Characteristics of Effective Tutoringscale and its TAKS Cumulative Met Standard for TAKS reading.
Research Question#2StatisticalMeasurementWhat do administrators reportabout the effectiveness ofimplementing Title I tutoringprograms in elementary schools asindicated by their rating on theCharacteristics of EffectiveTutoring Scale?Descriptive statistical measuresincluding a frequency polygon andgrouped frequency distribution willbe used to summarize the results ofthe survey.
Research Question#3StatisticalMeasurementWhat do elementary schoolteachers report about theeffectiveness of implementingTitle I tutoring programs inelementary schools as indicatedby their rating on theCharacteristics of EffectiveTutoring Scale?Descriptive statistical measuresincluding a frequency polygon andgrouped frequency distributionwill be used to summarize theresults of the survey.
DESCRIPTIVE STATISTICS• To determine any emerging trends, a grouped frequency distribution willbe prepared for each of the 10 schools involved in the study.• Participants’ scale scores will be placed in rank order from high to low.Visual representation of the data will be graphed in a frequency polygonin which the shape of the distribution of scores demonstrates theskewness of the data.• Descriptive data will be useful in reporting whether respondents rate theirschools on the higher end (effective) or the lower end (in need ofimprovement).• Measures of central tendency will also be tabulated to summarize the datapresented in the frequency distribution. The arithmetic mean will becalculated by adding up each of the respondent’s scores and dividing bythe number of scores (n=220).• For each school, the mean of all respondents’ scores will be calculatedin order to determine the overall level of effectiveness of each school(highly effective, efficient, emergent, needs improvement).• In addition, the mean of the scores from all 10 campuses will becalculated to ascertain the average level of effectiveness for Title Itutoring programs.
Research Question#4Hypothesis IndependentVariableIndependentVariableStatisticalMeasurementIs there adifference inthe ratings ofadministratorsand teachers ontheCharacteristicsof EffectiveTutoring Scale?There is nodifference inthe ratings ofadministratorsand teachers intheir overallscores on theCharacteristicsof EffectiveTutoring Scale.Administrators’scores on theCharacteristicsof EffectiveTutoring ScaleTeachers’scores on theCharacteristicsof EffectiveTutoring ScaleT-test ofIndependentMeans
INFERENTIAL STATISTICS• The second hypothesis, H2, involves inferential research.Statistical procedures used in the data analysis of the research willenable one to draw conclusions regarding the impact of freetutoring.• A t-test of independent means will be applied to determinewhether there is a significant difference in the responses ofadministrators and staff member on the Characteristics ofEffective Tutoring Scale.• Rating scale scores of administrators and faculty members will belisted into two separate columns for analysis in SPSS.• Using SPSS 15.0, a t-test will be calculated for the two sets ofscores.• To determine the statistical significance, the null hypothesis willbe restated.• An independent-means t-test will be applied at the standard alphalevel of .05.
In Conclusion…• Research has provided little evidence to guide policymakers and educators on the benefits of supplementaleducational services, particularly in improving theeducation of low-income and some minority students(Sunderman, 2006).• The study will fulfill the gap in the research to date byinvestigating whether administrators and teachers ratethe tutoring programs implemented on their campusesas effective or in need of improvement.• Whether the level of effectiveness of tutoring programsis related to student achievement will also beinvestigated.