• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD
 

A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD

on

  • 2,200 views

A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD ...

A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD


PhD Committee Members - Dr. M. Paul Mehta, Dissertation Chair; Committee Members: Dr. William Allan Kritsonis, Dr. Douglas S. Hermond, Dr. David E. Herrington, Dr. Camille Gibson

Statistics

Views

Total Views
2,200
Views on SlideShare
2,200
Embed Views
0

Actions

Likes
0
Downloads
21
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKES TESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS by Arhtur L. Petterway, PhD Document Transcript

    • A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKESTESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS ARTHUR L. PETTERWAY Submitted to the Graduate School Prairie View A&M University in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY May, 2007
    • A MIXED-METHOD ANALYSIS OF THE IMPACT OF HIGH STAKESTESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXAS A Dissertation by ARTHUR L. PETTERWAYApproved as to style and content by: L. M. Paul Mehta, Chair HO Aid- ie. Dr. Will itsonis . Dou las Hermond ck_ Dr. David Herrington Dr. Camille ibson F Dr. . Paul M Dean, The Whitlowe R. Green e of Education I Dr. William Parker Dean, Graduate School May 2007
    • ABSTRACT A Mixed - Method Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas April, 2007 Arthur L. Petterway: B.A. – Dillard University M.Ed., Prairie View A&M University Dissertation Chair: Dr. M. Paul Mehta Ample research has been conducted on the intrinsic validity ofstandardized assessments, and on the factors affecting the assimilationand integration of English language learners (ELLs). The reliability ofthese assessments as a universal tool to measure student learning, andas a basis for determining school performance needed closerexamination. The purpose of this study was to determine the impact of high-stakes testing on ELLs. This was shown in both the quantitative andqualitative dimensions of the study. Data obtained from Texas EducationAgency (TEA) were used to determine whether there was a relationshipbetween the percentage of ELLs enrolled in a school and the percentageof all students passing the 10th grade Texas Assessment of Knowledgeand Skills (TAKS) tests in the core areas of English Language Arts andMathematics given in 2003, 2004, 2005, and 2006. The qualitativeaspect of this study explored what certified English as a SecondLanguage( ESL) teachers, non-certified ESL teachers who teach ELLs, iii
    • administrators, and district ESL personnel viewed as the impact thathigh stakes standardized assessments had on ELLs, ESL curriculum,and instruction in ESL classrooms. This study determined the impact of high-stakes testing on ELLsusing the explanatory design of mixed method analysis. Data of 173major urban high schools obtained from the Texas Education Agency(TEA). It was determined through the Pearson correlation computationsusing the Statistical Package for the Social Sciences (SPSS) that therewas a significant relationship between the percent of ELLs enrolled in aschool and the percent of all students passing the 10 th Grade TAKS testsin English Language Arts and Mathematics. In the qualitative portion ofthe study, the views and opinions of district ESL personnel weregathered. Principals, assistant principals, ESL and non-ESL teacherstook part in an on-line, open-ended questionnaire; one-on-oneinterviews; and focus groups. The focus groups addressd the purposesof statewide testing; its intended consequences; problems and changescreated by TAKS, and the recommendations to improve ESL curriculumand instruction. The results of the study affirmed the expected outcome that asignificant relationship existed between the percentage of ELLs enrolledin a school and the percentage of all students passing the 10 th gradeTAKS tests in both core areas of English Language Arts andMathematics. The regression analysis predicted that as the percentage of iv
    • ELLs in a school increased, the performance on the statewide, high-stakes testing in terms of all students passing the 10th grade TAKS testsdecreased. Respondents of the study considered TAKS as a tool to gaugeknowledge in the different core areas. English language learners wereexpected to have at least average scores on TAKS. There was a differencein the expected and actual results; respondents observed dismal orfailing performance of ELLS in the actual results in TAKS. This wasevident by the high failure rate of ELLs in their respective schools. Higherdropout rate and lower graduation rate of ELLs were problemsencountered due to TAKS. Respondents favored a different test for ELLs,possibly given at a later date after ELLs had studied in the country for atleast several years. Respondents believed that interventions were neededto help ELLs perform better. Both the school and the home, together withthe community, have to be involved in preparing ELLs for their presentand future roles in the American society. Results of this study may provide valuable data to district andschool administrators to develop strategies that will improve theperformance of ELLs on the statewide, high-stakes testing and to developassessments that truly measure learning without the nullifying effect oflinguistic and cultural bias. The study may also help to enhance thereliability of standardized assessments as a tool to determineaccountability for student performance. v
    • DEDICATION This dissertation is dedicated in humble gratitude to my Lord and Savior, Jesus Christ, in whom I move, trust, and have my being. Through HIS divine wisdom and purpose, HE gave me my parents: Bob Stevenson Petterway November 23, 1923-September 28, 1992 and Myrtice Lee Petterway February 10, 1927-February 15, 1959They are now in Glory with HIM sharing this divine blessing that HE has bestowed upon me. vi
    • ACKNOWLEDGEMENTS I would like to acknowledge those without whom this work wouldnot have been possible. First and foremost I wish to acknowledge theblessings bestowed upon me by my Lord and Savior Jesus Christ.Among these blessings are the kind souls I mention here. First, I would like to acknowledge my dissertation committee,Dr. M. Paul Mehta, Dr. William A. Kritsonis, Dr. Douglas Hermond,Dr. David Herrington, and Dr.Camille Gibson. They have been thorough,fair, understanding, demanding, and, most of all, dedicated to academicexcellence in all phases of this work. I would especially like to thankDr. M. Paul Mehta for taking time, along with his duties as the Dean ofEducation, and carrying the baton of being my committee chairman andDr. Robert Marshall, my former committee chairman (who would not letMe fall off of my bicycle and taught me how to eat an elephant) for thelong hours they have spent and the endless patience they have shown asthey have guided me through this endeavor. I also wish to thankDr. Kritsonis for his passion for making sure that I get published before Ireceive my PhD and taking on the duties of head cheerleader for thisproject. Thanks goes to Dr. Hermond for serving as lead statistician andfor not allowing me to attempt to eat the whole pie and limiting me to asmall slice. Many thanks also to Dr. Herrington who guided me in tyingup several loose ends. I am also grateful to Dr. Gibson for taking time outof her very busy schedule to offer her support and encouragement. vii
    • Additional thanks go to Dr. William Parker, Dean of Graduate Studies,for believing in me and giving me the opportunity to prove myselfbeginning with my pursuit of my Masters Degree. I also gratefully acknowledge all of my professors and thank themfor the wisdom and knowledge they have so generously shared. I wouldfurther like to thank all of the faculty and staff of Prairie View A&MUniversity who have contributed to my achievement in countless ways. Ialso wish to acknowledge the unwavering encouragement of my studentcohort as we shared the joy and pain of this incredibly challengingpursuit. I would like to thank my principal, Mrs. Linda Llorente, for herunderstanding and support as I pursued this dream. She has beenabundantly patient and understanding of the demands this work hasplaced on my time and energy. I also wish to acknowledge thecontributions of my peers and colleagues at Austin High School.Although it is virtually impossible to name all who have contributed tothe completion of this work, I feel that there are several who must bethanked by name. I would like to thank Andy Lamboso and RhodoraMaligad who helped with proofreading and typing along with Kathy Koch,Betty Shaw, Debbie Kubiak, and Raul Asoy who helped with proofreadingand editing. Finally, I wish to acknowledge the precious prayers offered by therighteous to strengthen and uphold me through this challenging time, viii
    • with a special thanks to my most avid prayer warrior, my aunt, Mrs. SinaGunnels, and her chicken with a snuff cup under its wing. ix
    • TABLE OF CONTENTS PageABSTRACT ..................................................................................... iiiDEDICATION ................................................................................. viACKNOWLEDGEMENT .................................................................. viiTABLE OF CONTENTS ................................................................... ixLIST OF TABLES .......................................................................... xivCHAPTER I. INTRODUCTION .......................................................... 1 Statement of the Problem....................................................... 3 Purpose of the Study ............................................................. 6 Research Questions ............................................................... 7 Quantitative .................................................................. 7 Null Hypothesis One ...................................................... 7 Null Hypothesis Two ...................................................... 7 Qualitative .................................................................... 7 Description of the Research Design ........................................ 8 Assumptions ......................................................................... 9 Limitations of the Study ........................................................ 9 Delimitations of the Study .................................................... 10 Definition of Terms ............................................................... 10 Significance of the Study ...................................................... 12 Organization of Study ........................................................... 13CHAPTER II. REVIEW OF LITERATURE ......................................... 15 ix
    • Page No Child Left Behind (NCLB) ................................................. 15 Historical Perspective .................................................. 15 Description of the Key Factors ..................................... 18 Expectations for Parents .............................................. 20 Response to NCLB ....................................................... 21 Adequate Yearly Progress (AYP) ............................................. 22 Purpose and Support to NCLB ..................................... 22 Changes and Updates.................................................. 24 AYP and Limited English Proficient (LEP) Students ................ 30 Definition of English Language Learners (ELLs) ........... 30 Issues and Other Considerations of LEP ....................... 34 High Stakes/Statewide Testing ............................................. 43 Principles of Testing Programs ..................................... 44 Accountability in Testing ............................................. 50 Effects of High Stakes Testing on Student Motivation .... 52 Other Considerations of Assessment on Testing ............ 56 Related Studies .................................................................... 59 Summary ............................................................................. 61CHAPTER III.METHODOLOGY. ...................................................... 63Introduction .................................................................................. 63 Research Questions .............................................................. 64 Quantitative ................................................................ 64 x
    • Page Null Hypothesis One .................................................... 64 Null Hypotheses Two ................................................... 65 Qualitative .................................................................. 65Research Methods ................................................................ 66Research Design................................................................... 67 Quantitative ................................................................ 68 Qualitative .................................................................. 68Pilot Study ........................................................................... 68Population and Sample ......................................................... 70 Quantitative ................................................................ 70 Qualitative ................................................................. 71Instrumentation ................................................................... 72 Instruments ................................................................ 72 Validity ....................................................................... 74 Reliability ................................................................... 74Research Procedures ............................................................ 75 Quantitative ................................................................ 75 Qualitative ................................................................. 75Data Collection and Recording .............................................. 76 Quantitative ................................................................ 76 Qualitative ................................................................. 76Data Analysis ....................................................................... 78 xi
    • Page Quantitative................................................................ 78 Qualitative .................................................................. 79 Summary ............................................................................. 80CHAPTER IV ANALYSIS OF DATA .................................................. 83 Findings .............................................................................. 85 Quantitative Research Question .................................... 85 Null Hypothesis One ..................................................... 86 Null Hypothesis Two ..................................................... 86 Qualitative Research Question ..................................... 100 Discussion ........................................................................ 141 Summary .......................................................................... 145CHAPTER V SUMMARY, CONCLUSION, IMPLICATIONS ANDRECOMMENDATIONS ................................................................. 147 Summary .......................................................................... 147 Conclusions ...................................................................... 150 Implications ...................................................................... 151 Recommendations for Further Study .................................. 154REFERENCES ............................................................................. 157APPENDICES .............................................................................. 177 Appendix A IRB .................................................................. 178 Appendix B Consent Form .................................................. 183 Appendix C Interview Questions .......................................... 186 xii
    • Page Appendix D On-Line Questionnaire ..................................... 191 Appendix E Letter to Participants ........................................ 196 Appendix F Request for Extant Data from T.E.A... ................ 198VITA ........................................................................................... 201 xiii
    • LIST OF TABLESTable Page4.1.1 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2003 10th Grade TAKS for English Language Arts and Mathematics .............................. 874.1.2 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2004 10th Grade TAKS for English Language Arts and Mathematics ............................... 874.1.3 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2005 10th Grade TAKS for English Language Arts and Mathematics ............................... 884.1.4 Means and Standard Deviations of ELLs Enrolled in School and All Students Passing the 2006 10th Grade TAKS for English Language Arts and Mathematics ............................... 884.1.1.1 Comparison of Results in 10 th Grade English Language Arts TAKS ......................................................................... 894.1.1.2 Comparison of Results in 10 th Grade Mathematics TAKS...... 904.2.1 Pearson Correlation: 2003 10 th Grade TAKS for English Language Arts and Mathematics............................................ 914.2.2 Pearson Correlation: 2004 10 th Grade TAKS for English Language Arts and Mathematics............................................ 914.2.3 Pearson Correlation: 2005 10 th Grade TAKS for English Language Arts and Mathematics............................................ 92 xiv
    • Table Page4.2.4 Pearson Correlation: 2006 10 th Grade TAKS for English Language Arts and Mathematics............................................ 924.2.5 Coefficients for Percentage of All Students Passing the 2003 10th Grade English Language Arts TAKS ........................ 934.2.6 Coefficients for Percentage of All Students Passing the 2003 10th Grade Mathematics TAKS ...................................... 944.2.7 Coefficients for Percentage of All Students Passing the 2004 10th Grade English Language Arts TAKS ........................ 954.2.8 Coefficients for Percentage of All Students Passing the 2004 10th Grade Mathematics TAKS ...................................... 964.2.9 Coefficients for Percentage of All Students Passing the 2005 10th Grade English Language Arts TAKS ........................ 974.2.10 Coefficients for Percentage of All Students Passing the 2005 10th Grade Mathematics TAKS ...................................... 984.2.11 Coefficients for Percentage of All Students Passing the 2006 10th Grade English Language Arts TAKS ........................ 994.2.12 Coefficients for Percentage of All Students Passing the 2006 10th Grade Mathematics TAKS .................................... 1004.3 Distribution of Respondents by Gender .................................. 1024.4 Distribution of Respondents by Age ........................................ 1024.5 Distribution of Respondents by Professional Position ............... 1034.6 Distribution of Respondents by Highest Degree Earned ........... 103 xv
    • Table Page4.7 Distribution of Respondents by Years of Experience in Education ............................................................................. 1044.8 Distribution of Respondents by Certifications Held .................. 1054.9 Why is TAKS Given as a Statewide Test to ELLs? .................... 1074.10 What are the Anticipated Results of Statewide Testing for ELLs? ............................................................................ 1114.11 What are the Actual Results of Statewide Testing for ELLs? ............................................................................ 1154.12 What are the Intended Consequences of TAKS for ELLs? ....... 1184.13 What Has Happened to ELLs Because of TAKS? .................. 1224.14 What Problems Have Occurred for ELLs Due to TAKS? ......... 1264.15 What Changes Have Occurred for ELLs Due to TAKS? .......... 1294.16 What Recommendations are suggested for Improvement of ELLs Performance on TAKS?............................................. 1324.17 What are the Recommendations, with Greatest Value, are offered for ELLs Success on TAKS? ................................. 136 xvi
    • CHAPTER I INTRODUCTION For years, the English language learners (ELLs) have beensubjected to educational systems that did not expect them to rise to thesame standards as their native English-speaking peers (Winograd, 2002).Although that it can take several years to acquire the second languageskills needed to be successful in school (Collier, 1989), too often Englishlanguage learners born in the U.S. are still in English as a secondlanguage (ESL) classes and far behind their grade level peers in thecontent areas by the time they reach high school (Freeman & Freeman,2002). One factor that should be considered in this failure to reach gradelevel requirements is that language may constitute an element of self-identity. It is possible that minority groups are insistent on retainingtheir ethnic language as their “first.” English proficiency then would be amere elective instead of an indispensable learning tool. If this is the case,schools are being held accountable for the consequences of a socio-cultural phenomenon that is beyond their limited powers to address. Public schools are under close scrutiny. Since they are supportedby public funds, there is an increasing demand for accountability. TheNo Child Left Behind Act (NCLB) now requires all students to beaccounted for in any state‟s assessment system, although that has notalways been the case (Abedi, 2004). School districts are now required to 1
    • 2clearly demonstrate they deserve, and effectively utilize public funding. Initself, this is not a disturbing trend. Institutions that are wholly or partlysupported by public funds should be accountable. This is essentially aconsequence of democracy. A government that is created by, and for thepeople, is so unlike an aristocracy that is not required to serve aconstituency beyond the guarantee of protection from marauders orinvading armies. The U.S. system of government empowers the state toundertake measures that guarantee the common good. This goes beyondthe guarantee of physical safety, since the term “common good” has awider application, and implies a calculated sensitivity to every citizen‟spursuit of happiness. While education is not categorized as afundamental right, it is perceived as primary among a bundle of valuesessential for every person‟s quest for self-fulfillment and happiness. Thisexplains why there is little argument about whether the governmentshould be involved in education at all, and whether this is an endeavorbetter left to the private sector (Abedi, 2004). The government‟s involvement in education opens a wide avenuefor the analysis and evaluation of results. In today‟s world, it is notenough that public schools have adequate facilities, although thisconstitutes one level of analysis. It is important that schools are safe andteachers are qualified, although in the hierarchy of priorities consideredfor evaluating schools, these outcomes are not standard. Schools arejudged principally based on the amount of learning that takes place in
    • 3their classrooms. As an internal act, the evidence of learning is analyzedfrom scores students obtain on standardized assessments. Institutions are now facing an ever-increasing demand foraccountability. There is pressure from every conceivable corner to makepublic schools accountable to their stakeholders. This means that it isnot enough for students to learn in school; it is equally important thatlearning should occur in ways that are measurable. If students areunable to demonstrate what they have learned, it is presumed that nolearning took place at all. The time when public schools are allowed tooperate without proven success is over. It is appropriate to inquire aboutthe valid manifestations of success and learning, and how they mayactually be measured. Cultural construct renders school rankings flawedto a certain extent since they become less accurate as a measure of thefaculty and administration‟s performance. Instead, they becomeunintended indicators of the ethnicity of the students to which schoolscater (Abedi, 2004). Statement of the Problem High stakes assessment systems are meant to bring attention tothe needs of ELLs, who are most at risk of not reaching the educationalgoals set for them (Anderson, 2004). But what results do statewideaccountability tests really produce for ELLs (Anderson, 2004)?Assessment systems usually produce both positive and negativeconsequences (Anderson, 2004). The positive and negative consequences
    • 4of assessments are what is called „washback‟ (Alderson & Wall, 1993), orhow the results of an assessment affect the stakeholders taking the test(Anderson, 2004). While quantifiable washback effects such as increased dropoutrates or increased referral to Special Education have been researched,assessment washback is more complicated than numbers alone can tell(Anderson, 2004). Students who qualify for Special Education may beallowed to take alternative assessments in lieu of the state assessmentssuch as the Texas Assessment of Knowledge and Skills (TAKS). It isinteresting to note that while the numbers of African-American andHispanic students are over-represented in Special Education, about eightto nine percent of ELLs are identified as receiving Special Educationservices in the United States (D‟Emilio, 2003; Zehler, Fleischman,Hopstock, Pendzick, & Stepherson, 2003). While these assessments arenot on grade level, schools are expected to demonstrate that, based onstudents‟ scores on alternative assessments, improvement in academicperformance is taking place. Data are needed that tell us more about the full range of intendedand unintended consequences occurring in schools today (Anderson,2004). Since school rankings affect student and faculty morale, theyserve more as a force for the preservation of the status quo than a forcefor improvement in student performance. A school that works hard toensure that learning occurs, and that its students progress academically,
    • 5but which has a large proportion of ELLs, will risk being ranked asunderperforming because the measure used to evaluate its performanceis blind to this important demographic reality. One way to get at these data is by talking with the stakeholders atthe schools. Educators are the ones who deal directly with the impact ofhigh stakes assessments, but are overlooked in research. While teachers‟opinions are often cited as anecdotal evidence that a problem exists,their expert observations often go unrecorded in any systematic way(Anderson, 2004). Standardized assessments are a measure for holding schoolsaccountable for student learning. At the present time, schools in Texasare ranked Exemplary, Recognized, Acceptable or Underperforming,depending on the performance of their students in the Texas Assessmentof Knowledge Skills (TAKS). This produces a vicious cycle sinceexemplary schools attract the best students who may leaveunderperforming schools to seek what is perceived to be a higher qualityof instruction in higher ranked schools. These labels tend to have a self-fulfilling effect, or at least they make it difficult for underperformingschools to achieve higher performance scores on standardized tests,since they face the additional burden of surmounting language barriersand a history of low performance. Related to this concern is the prevailing system of voluntarysegregation in most zones and districts. Some schools have either a
    • 6predominant population of White, Hispanic, or African-Americanstudents. Each of these student groups is given the same tests, and yetthey have varying degrees of proficiency in the language in which theassessments are given. It begs to be asked whether these assessments,in fact, measure learning and whether they are linguistically andculturally neutral. The implication is that these students will be able toanswer the test questions even if they do not have equal exposure tocultural references that may frame some of the test questions. This study is intended to explore what educators perceive as theconsequences of statewide assessment for ELLs and what they observeas actually occurring (Anderson, 2004). Purpose of the Study The purpose of this study was to determine the impact of high-stakes testing on ELLs. This was shown in both the quantitative andqualitative dimensions of the study. Data obtained from TEA were usedto determine whether there is a relationship between the percentage ofEnglish language learners enrolled in a school and the percentage of allstudents passing the 10th Grade TAKS tests in the core areas of EnglishLanguage Arts and Mathematics given in 2003, 2004, 2005 and 2006. Tosupport the quantitative aspect, this study explored what certified ESLteachers, non-certified ESL teachers who teach ELLs, administrators,and district ESL personnel viewed as the impact that high stakes
    • 7standardized assessments have on ELLs, ESL curriculum andinstruction, and what they observed as actually occurring. Research QuestionsQuantitative Is there a relationship between the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005, and 2006? HypothesesH01: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005, and 2006?H02: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005, and 2006?Qualitative The major question addressed by this study was: What are theanticipated and observed consequences of the statewide testing,specifically TAKS, on ESL curriculum and instruction as viewed bycertified ESL teachers, non-certified ESL teachers who teach ELLs,school administrators, and district ESL personnel?
    • 8 This major question was explored using the following probes:1. Why is TAKS given as a statewide test?2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?)3. What problems have occurred related to or because of TAKS?4. What changes were caused by this statewide testing?5. What are your recommendations to improve this statewide testing?6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? Description of the Research Design The study analyzed the issues and challenges faced by ELLs andthe public schools that serve them. Quantitative data for this researchwere gathered from the Texas Education Agency (TEA) regarding thepercentage of ELLs and the performance of 10th grade students from themajor urban high schools in Texas on the Texas Assessment ofKnowledge and Skills (TAKS) tests in English Language Arts andMathematics for 2003, 2004, 2005, and 2006. Qualitative data werederived from one-on-one and focus group interviews and an on-linequestionnaire focusing on respondents‟ views and opinions about thevarious ways that standardized assessments impact ELLs.
    • 9 Assumptions Fraenkel and Wallen (2003) stated that an assumption is anythingtaken for granted rather than tested or checked. This study is nodifferent and the following assumptions were made: (a) that the firstlanguage of the ELLs is Spanish and they have varying degrees of fluencyin the English language; (b) that the ESL curriculum is appropriate forthe mastery of the TAKS test for the ELLs; (c) that the on-line open-endedqualitative questionnaire will be completed by the respondents on time;and (d) that the respondents in the focus groups will truthfully expresstheir views and opinions regarding issues or concerns brought to thegroup. Limitations of the Study Limitations of the study included several factors: mainly thequalitative questionnaire and the manner in which respondents gavetheir responses. The questionnaire may have vague questions open tomore than one interpretation. The pilot study helped in streamlining thequestionnaire to remove or modify such vague issues or concerns.Another limitation may have been the manner in which the respondentsanswered the question. For one reason or another, they may not havetruthfully answered some of the questions. The respondents may or maynot have completed the questionnaire due to no ready access to acomputer or they just did not want to complete the questionnaire. Thesenon-respondents became part of the mortality factor involved in the
    • 10study. Responses to the open-ended questions became difficult to classifyunder a certain category. This was facilitated through the Non-Numerical, Unstructured Data, Indexing Searching & Theorizing Vivo-“Nudist Alive” (NVivo) software system (Version 7.0) and by the focusgroup interviews where the respondents helped determine the category ofsuch responses. A factor that may have been encountered in the quantitativedimension of the study was the lack of intended data for the study.Diligent efforts were made to gather data from available sources. Delimitations of the Study The questions for the on-line qualitative questionnaire may havebeen a delimitation of the study. The pilot study contributed to theimprovement of the qualitative tool. Another delimitation may have beenthe choice of participants, especially in the focus groups. The “snowballtechnique” addressed this issue. Better interaction happened with added„quality‟ members to the focus groups. Qualitative data are available and the inclusion of the quantitativeaspect of the study provided a challenge and an opportunity to determineif certain factors of the study have any impact on the ELLs. Definition of TermsContent Standards are broad descriptions of the knowledge, skills, andunderstandings that schools should teach and students should acquirein a particular subject area (McLaughlin & Shepard, 1995).
    • 11English Language Learners (ELLs) is the preferred term to describe astudent whose native language is other than English (Chamot &O‟Malley, 1994). These students require instructional modifications, andeventually take the TAKS after two years of enrollment in the schooldistricts.High Stakes Assessment is an assessment in which student promotion(i.e., high school graduation) can be denied if the scores do not reflectcompetence (NCBE, 1997).Limited English Proficient (LEP) refers to a student with a languagebackground other than English, and whose proficiency in English is suchthat the probability of academic success in an English-only classroom isbelow that of an academically successful peer with an English-languagebackground (CCSSO, 1992).No Child Left Behind Act (NCLB) of 2001 (PL – 107 – 110). It is thereauthorization of the Elementary and Secondary Education Act (ESEA).Opportunity-to-learn (OTL) Standard defines the level and availability ofprograms, staff and other resources sufficient to enable all students tomeet challenging content and performance standards (McLaughlin &Shepard, 1995).Performance Standards are concrete examples and explicit definitions ofwhat students have to know and be able to do to demonstrate that suchstudents are proficient in the skills and knowledge framed by the contentstandards (McLaughlin & Shepard, 1995).
    • 12Standardized Assessments include the Texas Assessment of Knowledgeand Skills (TAKS) and the State and Locally-Developed AlternativeAssessment (SLDAA) for students who are exempted from the TAKS. Astandardized assessment is a measurement of what students know andcan do (McLaughlin & Shepard, 1995).Standards-based Reform requires setting standards of performance inacademic subject areas as a means of improving the substance of schoolcurricula and increasing the motivation and effort of students, teachers,and school systems and thereby improving student achievement(McLaughlin & Shepard, 1995).Washback shows how the results of an assessment affect thestakeholders taking the test (Alderson & Wall, 1993). Significance of the Study Expected outcome of this study may possibly provide additionalvaluable data for writers or researchers in regard to biases instandardized assessments that may encourage school districts to developassessments that truly measure learning without the nullifying effect oflinguistic and cultural bias. Additionally, this study enhances thereliability of standardized assessments as a tool in determiningaccountability where the performance of English language learners isconcerned.
    • 13 Organization of the Study Chapter I identifies the problem this study addresses: the impactof high stakes assessments on the curriculum and instruction of Englishlanguage learners. It includes the hypotheses and research questions ofthe present study. Included are the definitions of terms valuable to thestudy. Chapter II includes the review of literature about the essentialconditions and factors regarding the NCLB Act, the AYP implications forconcerned schools, high-stakes, statewide assessments and theimplications and challenges they present to the preparation andeducation of ELLs. The information reveals the difficulties that Englishlanguage learners face when taking these high stakes assessments, thepossible positive and negative consequences and possible “washback”related to the assessments. A mixed methods study is identified and expounded in Chapter III.Quantitative data for this research were gathered from the TexasEducation Agency regarding the percentage of ELLs and the performanceof major urban high schools in Texas in the statewide test (TexasAssessment of Knowledge and Skills) for 2003, 2004, 2005, and 2006.Qualitative data were derived from an on-line, open-ended questionnaireand interviews that focused on the respondents‟ views and opinionsabout the varied ways standardized assessments impact Englishlanguage learners.
    • 14 Results of the study are presented in detail in Chapter IV.Quantitative results include the available data collected from TexasEducation Agency. Results of computations employing the StatisticalPackage for the Social Sciences (SPSS) statistical package, (Version 14.0)are shown in tabular presentations and explanations regarding therelationship among the variables are included. Qualitative results includethe participants‟ views and opinions on the impact of high stakes testingon English language learners and the information collected from the on-line, open-ended questionnaire, individual and focus group interviews. Major findings of the study are discussed in Chapter V. Impact ofhigh stakes standardized assessments on English language learners arealso summarized. Other relevant factors that influenced this study arepresented, as well as recommendations for future research.
    • CHAPTER II REVIEW OF LITERATURE Key issues and concerns about the No Child Left Behind Act (NCLB) of2001 and the Adequate Yearly Progress (AYP) are major parts of the reviewof related literature. Included are the principles and accountability involvedin high-stakes testing and the descriptions and accommodations given tothe ultimate beneficiary of the efforts exerted by the federal and statepolicymakers, the school and district administrators – the learners,specifically, the English language learners who strive to be better citizens ofthis country. Short description of related studies on statewide testing andEnglish language learners (ELLs) are given to show their tie-in with thisstudy. No Child Left Behind (NCLB)Historical Perspective The NCLB Act of 2001 (PL – 107 -110), is the reauthorization of theElementary and Secondary Education Act (ESEA). The ESEA was firstpassed in 1965 with the goal of improving the U. S. educational system byproviding better education for students in poverty through an increase inservices to them. The ESEA provided federal funds for schools but did notrequire accountability in the use of those funds. In 2003, the Center ofEducational Policy clarified why accountability was not part of ESEA in1965: “At that time, the federal role in education was marginal, most stateeducation agencies had very limited authority and capabilities, and local 15
    • 16people were extremely wary that more federal aid would bring federalcontrol” (p.iv). The National Assessment of Educational Progress (NAEP) was initiatedas a federal testing program at about the same time when ESEA came intoexistence. NAEP was tasked to report how the nation‟s students wereperforming on selected items at the three grade levels --- 4th, 8th and 12th.Brennan (2004) reported that there were fears that the NAEP might becomea “high-stakes federal testing program” found in some European countries.He explained that, “to help preclude that possibility, it was written into lawthat NAEP could not report scores for individual students” (p.2). The NAEPevolved through the 1980s and early 1990s from a reporting of item scoresto test scores and then, on a trial basis, to a reporting of scores thataddressed achievement levels (below basic, basic, proficient, and advanced).It is currently used to confirm state NCLB testing results which, accordingto Brennan, “is the de facto elevation of NAEP to a federally-mandated high-stakes testing program” (p.9). Through the NCLB Act, policymakers in Washington seek to raiseacademic achievement in the nation by requiring schools to assess allstudents on specified content areas and report their progress towardproficiency. Focus of NCLB is on core academic subjects as defined in thelaw: “The term „core academic subjects‟ means English, reading or languagearts, mathematics, science, foreign language, civics, and government,
    • 17economics, arts, history, and geography” (U.S. Department of Education,2002). The premise of NCLB is that our nation‟s schools are failing. Thus, thepurpose of NCLB is raising the achievement of all students and eliminatingthe achievement gap among students differentiated by race, ethnicity,poverty, disability, and English proficiency. Since this Act redefines thefederal role in education policy that has traditionally been a stateresponsibility, it merits the attention of educators, parents and citizens.Because the NCLB Act has an impact on the teaching and the learning ofthe core content areas, including languages, language educators need to beinformed about it. If a roomful of educators were asked which word or phrase best sumsup No Child Left Behind (NCLB), many would say accountability. Othersmight propose student achievement, proficiency or raised expectations. Butperhaps the most accurate word to encapsulate the United States‟ mostambitious federal education law – which proposes to close achievement gapsand aims for 100% student proficiency by 2014 - is testing. Certainly, thefocus on holding schools accountable for student achievement onstandardized tests sets NCLB apart from previous versions of the law.(Guilfoyle, 2006).
    • 18Description of the Key Factors There are four key elements in the NCLB Act (Rosenbusch, 2005): (1) Accountability. States are required to establish a definition ofstudent proficiency in the core academic subjects of Reading/LanguageArts, Mathematics and Science through prescribed indicators and set atimetable to bring all students in all subgroups up to the defined levels ofproficiency by 2013-2014. The school must report to parents their child‟sprogress in each targeted academic subject annually, and the state isrequired to report the results of students‟ performance on the annual testsfor every public school to parents and the community. Schools that fail tomeet state-defined AYP toward their defined goals for two years areidentified as needing improvement. Schools that have not met AYP after fouryears are subject to restructuring or reconstitution. (2) Testing. States must develop and administer annual tests thatdefine the proficiency that all students are expected to reach inReading/Language Arts, Mathematics, and Science. States must include asample of students in fourth and eighth grades in a biennial NAEP inMathematics and Reading to verify state assessments. NCLB requires that by School Year (SY) 2005-2006, each state mustmeasure every child‟s progress in Reading and Mathematics in each ofgrades 3 through 8 and at least once during grades 10 through 12. In themeantime, each state must meet the requirements of the previous lawreauthorizing ESEA (the Improving America‟s Schools act of 1994) for
    • 19assessments in Reading and Mathematics at three grade spans (3-5; 6-9;and 10-12). By SY 2007-2008, states must have in place Scienceassessments to be administered at least once during grades 3-5; grades 6-9;and grades 10-12. States must ensure that districts administer a test ofEnglish proficiency to measure oral language, Reading and Writing skills inEnglish to all limited English proficient students, as of SY 2002-2003.Students may still undergo state assessments in other subject areas (i.e.,History, Geography, and Writing skills), if and when the state requires it.NCLB requires assessments only in the areas of Reading/Language Arts,Mathematics, and Science. (3) Teacher Quality. Public elementary and secondary school teacherswho teach core content areas are required to be “highly qualified,” which isdefined as having full state certification (may be attained through alternateroutes specified by the state), holding a bachelor‟s degree, and havingdemonstrated subject matter competency as determined by the state underNCLB guidelines. States are required to develop a plan by the end of 2005-2006 to ensure that every teacher is highly qualified to teach in his or hercore content area. (4) Scientifically-Based Research. The NCLB Act requires that alleducational decisions be informed by scientifically-based research asdefined in the legislation. The NCLB Act funds for Reading First Grants, forexample, are to be used for methods of reading instruction backed byscientifically-based research.
    • 20Expectations for Parents Due to NCLB (from Collegeboard.com) (1) New standards for students will require that beginning 2005,students in grades 3 through 8 must be tested in Mathematics and Englishto ensure they are meeting state standards. Students in Grades 10 through12 will be tested at least once. By 2007, states will begin testing students inScience as well. Results of the yearly tests will be known to parents. NCLBrequires that school districts provide parents with an annual “report card”that shows how well students in each school performed. The information isbroken down by race, ethnicity, gender, disability status, and othercategories so that parents will know how well each school is doing ineducating minority students or those with disabilities. (2) By the end of SY 2005-2006, teachers must be “highly qualified” inthe subjects they teach. States will determine what skills teachers musthave to be “highly qualified”, but the requirements could include a degree inthe subject they teach or extra training. States must provide annual reportcards about teacher certifications, including the percentage of classrooms inthe state not taught by highly qualified teachers. Principals must alsomaintain information about whether or not their school‟s teachers meet therequirements. (3) Each year, schools must increase the number of students whoachieve state standards. At the end of 12 years, all students should be ableto pass the tests. Schools that fail to achieve this progress will be targeted
    • 21for improvements that could include increased funding or staff andcurriculum changes. (4) NCLB requires school districts to notify parents if the child‟sschool has been identified as needing improvement as a result of failing toincrease the number of students meeting state standards. (5) About half of all public schools receive funding to help studentsfrom low-income families. If such a school is targeted for improvement andfails after two years, parents can choose to transfer their child to anotherschool or enroll in free tutoring. Parents have this choice for as long as theschool fails to adequately perform.Response to NCLB (Rosenbusch, 2005) NCLB has engendered controversy that is centered in part on theincreased role of the federal government in educational policy. A majority ofAmericans believe that decisions about what is taught in public schoolsshould be made at the local level by the school board (61%), rather than atthe state level (22%) or the federal level (15%) (Rose & Gallup, 2003).Results of a 2004 survey indicate that they disagree with “the majorstrategies NCLB uses to determine whether a school is or is not in need ofimprovement” (Rose & Gallup, 2004, p.2). For example, 83% of thosesurveyed believe that testing only in English and Mathematics will not yielda fair picture of the school, 73% say it is not possible to judge a student‟sproficiency in English and Mathematics on a single test, and 81% areconcerned that basing decisions about school on students‟ performance in
    • 22English and Mathematics will mean less emphasis on art, music, historyand other subjects. In the U.S. Department of Education, there is support for highstandards and high expectations for every child, but the NCLB focus onstandardized testing is resulting in a narrowing of the curriculum and a“sorting of students” (Marshak, 2003, p.229) and “could halt thedevelopment of truly significant improvements in teaching and learning”(Lewis, 2002, p.179). The National Education Association supports theNCLB Act in its goal but views it as an obstacle to improving publiceducation because of its focus on “punishment rather than assistance”, and“mandates rather than support for effective programs” (National EducationAssociation, n.d.). Adequate Yearly Progress (AYP)Purpose and Support to NCLB The No Child Left Behind Act of 2001 (NCLB; Public Law No. 107-110,115 Stat. 1425, 2002), the most recent reauthorization of the Elementaryand Secondary Act of 1965, holds states using federal funds accountable forstudent academic achievement. States are required to develop a set of high-quality, yearly student assessments that include, at a minimum,assessments in Reading/Language Arts, Mathematics and Science. Eachyear, they must report student progress in terms of percentage of studentsscoring at the “proficient” level or higher. This reporting is referred to asadequate yearly progress (AYP). A state‟s definition of AYP should also
    • 23include high school graduation rates and an additional indicator for middleschools to reach the “proficient” level or higher, which must be no morethan 12 years after the start date of the 2001 – 2002 school year, providedthat the first increase occurs within the first 2 years (Abedi, 2004). AYP will be reported for schools, school districts, and the state for allstudents. In addition, AYP must be reported for the following subgroupcategories of students: (a) economically disadvantaged students, (b)students from major racial and ethnic groups, (c) students with disabilities,and (d) students with limited English proficiency (LEP). According to theeducational statistics for 2000 – 2001 school year, the total number ofstudents labeled as LEP in the nation‟s public schools is more than 4.5million or 9.6% of total enrollment; (National Center for Education Statistics[NCES], 2002). States are continuing to find new ways to calculate AYP under theNCLB, in order to increase the number of schools and districts that meetthe student achievement targets set by law. Over the past few years, theU.S. Department of Education (ED) has allowed states to make manychanges in the way they determine AYP, including the following: (1)confidence intervals, which make allowances for natural fluctuations in testscores and essentially bolster a school‟s or subgroup‟s percentage ofstudents scoring at proficient levels; (2) performance indices that allowschools to get “partial credit” for the performance of students below theproficient level; (3) retesting, which allows students to retake a different
    • 24version of the same test and permits schools to use a student‟s best score tocount toward AYP, and (4) increased minimum subgroup sizes, which meanthat in many schools, subgroups do not get counted for AYP purposes. Thechanges have the effect of making it easier for the schools to make AYP,early indications are that the number of schools not making AYP has leveledoff, despite predictions that this number would increase as proficiencytargets rose (Olson, 2005).Changes and Updates In NCLB‟s original conception, determining AYP for a subgroup ofstudents, a school, or a district was already fairly complicated. States had toestablish, for every year between 2003 and 2014, a set of ever-increasingstate targets in terms of the percentage of students scoring at the proficientlevel or above on annual tests, with a final goal of 100% proficiency in 2014.If at least 95% of the students in each subgroup are tested, and if allstudents and subgroups meet the state proficiency targets, the school ordistrict makes AYP. The school has to meet targets for an additionalacademic indicator, such as the graduation or attendance rate. The law hasa “safe harbor” provision: if a school or subgroup fails to meet the statetargets, it could still make AYP if it reduces the number of students who arenot proficient from the previous year by 10%, and meets its additionalacademic indicator. Some other state changes that have been approved are brieflysummarized below (Center on Education Policy, 2005):
    • 25 Minimum subgroup size. To make AYP, schools and districts mustmeet achievement targets for each significant subgroup of students enrolled,such as African-American students, low-income students, or students withdisabilities. Higher minimum subgroup sizes mean that in many schools,subgroups do not get counted for AYP purposes. Thirteen states increased their minimum subgroup sizes in 2004; tenmore did so in 2005. The trend is away from a single minimum size andtoward larger subgroup sizes, different subgroup sizes for differentsubgroups and/or purpose, and the use of formulas for determiningsubgroup sizes. Georgia is one state that uses a formula approach. Itssubgroup size varies according to the size of the school; the minimum size iseither 40 students or 10% of a school‟s student population, whichever isgreater, with a cap of 75 students. Participation averaging. NCLB requires 95% of the students in everyschool and every subgroup within a school to take each subject testrequired by the Act. If this test participation requirement is not met, theschool cannot make AYP even if its test scores meet state targets. In March2004, the Department relaxed this requirement, allowing states to averagetheir participation rates over two or three years, so that a 94% participationrate one year could be balanced by a 96% participation rate the following orprevious year. In 2005, six states changed their accountability plans toincorporate this new policy, in addition to the 32 that did so last year.
    • 26 English language learners. Initially the U.S. Department of Education(ED) required all English language learners to be tested with the samegrade-level tests as other students. In response to state and local criticism,the Department revised its policy in February 2004 to allow states to exemptimmigrant students who are in their first year of enrollment in a U.S. schoolfor less than one year from taking the regular state English Language Artstests. These students still have to take an English language proficiency testand a Mathematics test, but the results need not count toward AYP. Whencalculating AYP for the subgroup of English language learners, states canalso count the progress of former English language learners for two yearsafter they reach English proficiency. Six more states adopted these changesin 2005, in addition to the 36 states that did so in 2004. Extra time is given for students with disabilities and English languagelearners to graduate. In 2005, eight states received approval from ED tocount students with disabilities and/or English language learners asgraduating on time even if they need extra years of high school. Seven statesreceived permission to do this in 2004. For students with disabilities, theirindividualized education plans would need to call for extra years of highschool beyond age 18. English language learners can be counted asgraduating on time if it takes five years, or as determined on a case-to-casebasis (Center on Education Policy, 2005). Identifying districts for improvement. In 2005, ED approvedamendments requested by 13 states to identify a district as being in need of
    • 27improvement only when it does not make AYP in the same subject andacross all three grade spans (elementary, middle and high school) for twoconsecutive years. In 2004, 18 states made this change. Californiaattempted to have ED accept a relatively lenient method that exempteddistricts where low-income students reached a certain level on state tests.ED rejected that method, and California settled on the grade span approachinstead (Davis & Sack, 2005). Annual measurable objectives. Eleven states changed their annualscore targets in 2005; four states did so in 2004. For example, Florida wasallowed to change its schedule of annual measurable objectives so thattargets would increase in smaller increments annually, rather than in largeincrements every three years (Olson, 2005); Virginia did so as well. Severalother states, including Alabama, Alaska, New Mexico, and North Carolina,changed their annual targets because they were introducing newassessments. NCLB is a demanding law. The achievement goals are ambitious, andthe burden on states and districts of declaring schools in need ofimprovement and then imposing sanctions on them is high. To try to meetthese demands, states have a strong incentive to keep the numbers ofschools and districts not making AYP as low as possible. Unable to changethe fundamental requirements written into the law, states are usingadministrative methods to lessen the numbers of schools and districts notmaking the AYP – confidence intervals, indexing, and other techniques.
    • 28 Education Secretary Margaret Spellings has been more flexible thanher predecessor in policies regarding students with disabilities, and ingranting special exemptions to some districts in the areas of school choiceand supplemental educational services (tutoring). Secretary Spellings hasdecided to allow the Chicago school district to provide tutoring despite thefact that the district has been identified for improvement (Gewertz, 2005).This exemption was then extended to New York City, Los Angeles, Boston,Memphis, Anchorage, and Dayton. This was a regulatory change. Secretary Spellings went further with four districts in Virginia bysuspending a key element of the law itself, invoking a clause in NCLB thatallows the Secretary of Education to do so. Her action exempted thesedistricts from the law‟s requirement that they provide school choice beforetutoring (Olson, 2005). Secretary Spelling‟s letter to Virginia officialsindicates that this is a pilot program intended to raise the numbers ofstudents receiving supplemental educational services (Spellings, 2005). Inaddition, districts in the five states most affected by Hurricane Katrina wereallowed to postpone, for one year, the consequences that follow when aschool is in need of improvement, such as tutoring, restructuring, andcorrective action (Olson & Davis, 2005). ED‟s willingness to make adjustments based on state and localexperience is commendable. But on the downside, parents in many stateswould now find it difficult to understand what it means when a school doesor does not make AYP, and what criteria were used to determine this
    • 29success or failure. For example, parents in Pennsylvania may see a reportcard that indicates that their child‟s elementary school has made AYP, butmight wonder whether the school is improving or whether it simply madeAYP as the result of what might be seen as a new “loophole” in the law. Theparents probably would not understand that the school may have made AYPthrough the use of a 95% confidence interval, safe harbor with a 75%confidence interval, or the Pennsylvania Performance Index as a second safeharbor. In other states, parents of English language learners, students withdisabilities, or other subgroups may not realize that raising the minimumsubgroup sizes means that their children no longer count for AYP purposesat the school level. They might not realize that the use of confidenceintervals allows for considerable leeway in a subgroup‟s test scores notavailable to larger groups of students, and that this is occurring despite theassertion that improving achievement for subgroups is a major focus of thelaw. Other drawbacks to the increasing complexity may contribute in thedifficulty of discerning clear trends in the number of schools and districtsnot making AYP, because the rules governing AYP keep changing every year.Amid these changes, it is impossible to determine whether an increase inthe number of schools making AYP within a state is due to better teachingand learning or NCLB rule changes. The constant rule changes, particularlythe use of large confidence intervals and ever-increasing minimumsubgroup sizes, may raise questions about whether the law is being watered
    • 30down so much that it shortchanges the very groups of disadvantagedchildren that it aims to help. Public support may wither if theimplementation of the law is perceived as deceptive or confusing. As states continue to learn from one another about the new types offlexibility that ED is allowing, and as state achievement targets continue torise until 2014, changes in AYP policies are likely to occur at a more rapidpace, at the expense of the public‟s ability to understand these changes.More transparency is needed at both the state and federal levels. Statesmust fully and clearly explain their rationales for requesting changes toaccountability plans. Once changes are approved by ED, they should beexplained in such a way that the public understand how AYP is determined. At the federal level, ED should more systematically and promptlypublicize its decisions about what types of changes to state accountabilityplans are and are not acceptable, and why. The current process of grantingchanges does not help state officials learn from other states‟ experiences,nor does it help them understand how ED is interpreting the intent of thelaw. Adequate Yearly Progress (AYP) and Limited English Proficient (LEP) StudentsDefinition of English Language Learners (ELLs) and LEP Limited English Proficient (LEP) students are students who lacksufficient English skills to participate in a regular education, all-Englishspeaking classroom. English Language Learner (ELL), according to Rivera
    • 31and Stansfield (1998), is a positive way to refer to any LEP student inEnglish. NAEP does not provide a definition of the LEP population; instead itpresents criteria for the inclusion of LEP students. NAEP inclusion criteriaindicate that: A student who is identified on the Administration Schedule asLEP and who is a native speaker of a language other than English should beincluded in the NAEP assessment unless: (a) the student has receivedReading or Mathematics instruction primarily in English for less than 3school years including the current year , and (b) the student cannotdemonstrate his or her knowledge of Reading or Mathematics in Englisheven with an accommodation permitted by NAEP (NCES, 2001). Due to the importance of LEP subgroups in NCLB accountability andreporting, NCLB provides an operational definition of LEP (NCLB, 2002).According to this definition: The term „limited English proficient‟, whenused with respect to an individual, means an individual (a) who is aged 3through 21; (b) who is enrolled or preparing to enroll in an elementaryschool or secondary school; (c) who was not born in the United States orwhose native language is a language other than English; who is a NativeAmerican or Alaska Native, or native resident of the outlying areas; and whocomes from an environment where a language other than English has had asignificant impact on the individual‟s level of English language proficiency;or who is migratory, whose native language is a language other thanEnglish, and who comes from an environment where a language other than
    • 32English is dominant; and (d) whose difficulties in speaking, reading, writing,or understanding the English language may be sufficient to deny theindividual the ability to meet the State‟s proficient level of achievement onState assessments described in section 111(b)(3); the ability to successfullyachieve in classrooms where the language of instruction is English; or theopportunity to participate fully in society. The term “English language learner” (ELL) is a recent designation forstudents whose first language is not English. This group includes studentswho are just beginning to learn English as well as those who have alreadydeveloped considerable proficiency. The term reflects a positive focus onwhat these students are accomplishing – mastering another language- andis preferred by some researchers to the term “limited English proficient”(LEP), the designation used in federal and state education legislation andmost national and state data collection efforts (August & Hakuta, 1997;LaCelle-Peterson & Rivera, 1994). The ELL population is highly diverse, and any attempt to describe thegroup as a whole, as with any diverse group of people, is bound to result ininaccurate generalizations. While this group of students share oneimportant feature - the need to increase their proficiency in English - theydiffer in many other important respects. ELLs are a diverse cross-section ofthe public school student population. The primary language, culturalbackground, socio-economic status, family history, length of time in the
    • 33United States, mobility, prior school experiences, or educational goals of anystudent in this group can distinguish him or her from any other ELLs. ELLs represent a rapidly growing, culturally and linguistically diversestudent population in the United States. In 2000-2001, LEP studentscomprised nearly 4.6 million public high school students. The majority wereSpanish speakers (79.0%), followed by Vietnamese (2.0%), Hmong (1.6%),Cantonese (1.0%), and Korean (1.0%). Since the 1990-1991 school year, thelimited English proficient population has grown approximately 105%, whilethe overall school population has increased by only 12%. English learners matriculate in schools throughout the nation, butmost frequently in large urban school districts in the Sun Belt states, inindustrial states in the Northeast, and around the Great Lakes. This trendis changing as immigrants move to more affordable suburban and ruralareas and to areas where language-minority families are relative newcomers,such as the Midwest. More than half (56.1%) reside in four states alone:California (32.9%), Texas (12.4%), Florida (5.6%) and New York (5.2%)(Kindler, 2002). English learners represent one in four K – 12 students inCalifornia schools (California Department of Education, 2000). This population includes recent immigrants as well as children bornin the United States. In the 2000-2001 school year, more than 44% of allLEP students were enrolled in Pre-K through Grade 3; about 35% wereenrolled in Grades 4 – 8; and only 19% were enrolled at the high school level(Kindler, 2002). Many LEP students attend schools where most of their
    • 34peers live in poverty. There are numerous differences among Englishlearners; for example, Spanish-speaking families tend to have lower parentaleducational attainment and family incomes than Asian-or Pacific-languagefamilies (August & Hakuta, 1997). Many criteria are used across the nation for identification of ELLs.Among the most commonly used criteria are Home Language Survey resultsand scores from English proficiency tests. There are reasons to believe thatthe Home Language Survey results may not be valid because of parents‟concern over equity in education for their children, parents‟ citizenshipissues, and communication problems (Abedi, 2004b). Similarly, there areconcerns about the validity of current English proficiency tests, such as theLanguage Assessment Scales and other commonly used assessments(Zehler, Hopstock, Fleischman & Greniuk, 1994). Criterion-related validitycoefficients, or the correlation between English proficiency tests and otherexisting valid measure of English proficiency, are not strong, explaining lessthan 5% of the common variance (Abedi, 2003). Finally, in terms of contentand construct validity, there is little evidence that the contents of theexisting English proficiency tests align sufficiently with commonly acceptedEnglish language proficiency standards, such as standards by Teachers ofEnglish to Speakers of Other Languages (Bailey & Butler, 2003).Issues and Other Considerations of LEP Disaggregated progress reports by subgroups mandated by the NCLBlegislation will monitor the nation‟s goal of having “no child left behind.”
    • 35However, there are major issues in this disaggregated reporting amongdifferent subgroup categories (students who are economicallydisadvantaged, students from major racial and ethnic groups, students withdisabilities, and LEP students). NCLB requirement for subgroup reportingmay give the impression that students in the subgroup categories start theachievement race at about the same level and can progress with otherstudents at about the same rate. This might be an overly optimistic view ofthe situation of less advantaged learners. By focusing this discussion onthe consequences for schools enrolling LEP students, we see how puttinginto practice the policy may produce invalid assessment and unreliablereporting while exacerbating the burdens of current educators. Following isa discussion of some challenges in AYP measurement and reporting for LEPstudents. Results of research on the assessment of LEP students suggest astrong confounding of language and performance. LEP students exhibitsubstantially lower performance than non-LEP students in subject areashigh in language demand. Studies suggest that the large performance gapbetween LEP and non-LEP may not be due mainly to lack of contentknowledge. LEP students may possess the content knowledge but may notbe at the level of English language proficiency necessary to understand thelinguistic structure of assessment tools. Strong confusion of languagefactors and content-based knowledge makes assessment and accountability
    • 36complex for LEP students and, very likely, students in other targetedgroups. Because of the strong effect of language factors on the instruction andassessment of LEP students, they lag far behind native English speakers.This leads to huge initial differences. LEP students start with substantiallylower baseline scores. More important, unless LEP students‟ Englishlanguage proficiency is improved to the level of native English speakers-which is not an easy task- they will not be able to move at the same rate onthe Adequate Yearly Progress line as do native English speakers. NCLB cannot have much of an effect on the initial performancedifferences between LEP and non-LEP students. A more sensible questionhere is whether or not NCLB can provide enough resources to schools with alarge number of LEP students to help them increase these students‟language proficiency to a sufficient extent that they can progress with theirnative English speaker peers in both instruction and assessment. Inconsistency in LEP classification across and within states makesAYP reporting for LEP students even more complex. If students are notcorrectly identified as LEP, how can their AYP be reliably reported at asubgroup level? Although NCLB attempts to resolve this issue by providing adefinition for this group, its criteria for classifying LEP students may facethe same problems as the existing classification system (Abedi, 2003;Zehler, Hopstock, Fleishman & Greniuk, 1994).
    • 37 Inconsistency in the classification of LEP students may lead to moreheterogeneity in the LEP subgroup. With a more heterogeneous population,larger numbers of students are needed to provide the statistically reliableresults required by NCLB. The population of LEP students in many districtsand states is sparse. In many states, there may not be enough students in adistrict or school to satisfy even the minimum number of 25 studentssuggested in the literature (Linn, Baker & Herman, 2002). Other researchershave argued that even 25 students may not be enough to providestatistically reliable results and have proposed a minimum group size of 100students (Hill & DePascale, 2003). Considering a small number of LEPstudents in many districts and states, the small group size for LEP reportingwould be another obstacle in regard to reliable AYP reporting. The LEP subgroup suffers from yet another major problem related toAYP reporting: The lack of stability of this group. In many states anddistricts across the nation, LEP students‟ level of English proficiency isreevaluated regularly, and if they reach a proficient level of Englishproficiency, they move out of the LEP subgroup. While this helps the moreEnglish-proficient students receive more appropriate instruction andassessment, it results in the LEP subgroup continuing to be low-performing.The students in this group will always be labeled as underachievers, andschools with large number of LEP students will be stuck in the “need forimprovement” category.
    • 38 Some states with substantial numbers of LEP students haveexpressed concern over this issue. They have proposed ideas and negotiatedwith the federal government to ease the level of possible negative impactthat this situation may have on school, district, and state accountability.For example, Indiana and Delaware will continue to include exited LEPstudents in the LEP subgroup for 2 years after they have been determined tobe proficient in English. Georgia plans to include LEP students as long asthey still receive services through the English for Speakers of OtherLanguages program, even if they have met exit criteria (Erpenbach, Forte-Fast & Potts, 2003). In California, students re-designated as LEP will remainin the LEP category until they reach the proficient or above level on theCalifornia Standards Test in English-language arts for 3 consecutive years(California Department of Education, 2003). However, the question ofwhether this policy will provide a long-term solution to the problem of LEPsubgroup instability or serve only as a temporary relief remainsunanswered. The measurement of the academic achievement of LEP students ismuch more complex than what the NCLB legislation conceives. A fairassessment of students in the four targeted subgroup categories requiresmuch more serious consideration than is outlined in the law. Despiteattempting to solve the age-old problem of heterogeneity among LEPstudents, the NCLB seems to perpetuate it, thereby leaving more room forchildren to be left behind.
    • 39 On the other hand, NCLB‟s attention to students in the four subgroupcategories in general and to the LEP population in particular is a step in theright direction. Considering that Title III of NCLB requires assessment ofLEP students‟ English proficiency on an annual basis and providing supportto states to develop reliable and valid measures of students‟ proficiency ispromising. Any decisions concerning assessment for all subgroups,particularly LEP students, must be informed by results of research andexperience in the education community. Currently, several tests for measuring students‟ level of Englishlanguage proficiency exist. Some of these tests have been used for manyyears by different states and districts. In spite of the existence of such tests,states are developing new English language proficiency tests with fundingthrough NCLB‟s Enhanced Assessment Instruments. A reasonableexplanation for this might be that states did not find that the existing testsprovided reliable and valid measures of students‟ level of English languageproficiency as required by NCLB. If this is the reason for the development ofthe new tests, then the test developers should be aware of problems in theexisting tests to avoid the same problems in the new tests. For example, a careful review of some of the most commonly usedlanguage proficiency tests concluded that the tests differ considerably intypes of tasks and specific item content and are based on differenttheoretical emphases prevalent at the time of their development (Zehler,Hopstock, Fleischman & Greniuk, 1994). This suggests that in the case of
    • 40some of the existing tests, the English language proficiency domain was notoperationally defined before the test development process. This and similarstudies and reviews should inform the development process of new tests.For example, it is imperative this domain be operationally defined before anyeffort in developing an English proficiency test. This definition should bebased on current developments in the areas of psycholinguistics,developmental psychology, education, linguistics, and psychometrics.Content standards for English for speakers of other languages should alsobe considered (Bailey & Butler, 2003). In analyzing data from the administration of existing languageproficiency tests, researchers have expressed concerns about the reliabilityand validity of these tests, the adequacy of the scoring directions, and thelimited populations on which test norms are based. For example, analysesof several large data sets from different locations across the nation haveshown validity problems in predicting LEP classification and lack of powerin identifying different levels of English language proficiency among the LEPstudent population (Abedi, 2003; Abedi, Leon, & Mirocha, 2003). Thoseinvolved in the development of new English language proficiency testsshould learn from such research and should conduct more analyses on thewealth of data that exists in this area. To be considered valid and reliablemeasures of English language proficiency, as outlined in the NCLB, newtests must first go through a rigorous validation process. Otherwise, there
    • 41may not be a reasonable justification to spend the limited NCLB resourceson English language proficiency test development (Abedi, 2003). As a final thought, assessment and accountability of LEP studentscannot be pursued in isolation of other important factors. An effectiveeducation system for LEP students that may lead to a successful AYPoutcome should include at least three interactive components: (a)classification, (b) instruction, and (c) assessment. A problem in any one ofthese components may affect the other two. For example, a studentmisclassified as LEP student may be assigned a different curriculum andthus receives inappropriate instruction. Alternately, inappropriateinstruction may result in low performance that may in turn result inmisclassification. While each component has a unique role, they sharecommon ground - the effect of language factors or barriers. Unnecessarylinguistic complexity of assessment may threaten the validity andequitability of assessment among LEP students. Complex linguisticstructure of instruction may negatively affect LEP students‟ ability tounderstand classroom instruction, and invalid assessment of students‟ levelof English proficiency may result in misclassification. In a positive light,valid assessment may provide diagnostic information that can informinstruction and classification (Abedi, 2003). An effective way to help LEP students reach proficiency in the AYPmodel is to consider the broader picture using the interactive model. Thefollowing are few critical needs:
    • 421. Improve current LEP classification and assessment. There is a need to establish a common definition of English language proficiency and substantially improve the validity of LEP instruments. Among other things, validity of LEP assessment can be enhanced by avoiding cultural biases and reducing unnecessary linguistic complexity of assessments.2. Improve monitoring of progress. Schools need effective and valid data collection methods that can be used to monitor LEP progress at every stage of a student‟s education. Weaknesses must be quickly addressed with appropriate instructional strategies.3. Improve teacher quality. LEP students need teachers who are well qualified in both language development and content, each of which plays a crucial role in LEP student achievement. The federal government can play a key role in this process by funding and encouraging programs that improve teacher capacity in this dual role. Teachers of LEP students should receive training in content delivery, language sheltering, and the teaching of the academic language.4. Consider redesignated LEP students as part of the LEP subgroup that established the baseline score. State plans allowing redesignated students to remain in the LEP subgroup for only a limited time are temporary fixes. While new LEP students are added to the subgroup, redesignated students should be retained for AYP reporting. This “semicohort” approach to tracking LEP students allows the progress of
    • 43 redesignated students to be counted toward subgroup AYP progress (Abedi, 2003). Based on the results of the research, policymakers, lawmakers, anddecision makers are urged to take appropriate action to correct theinequities resulting from the NCLB in regard to the subgroups targeted bythe legislation, particularly the LEP student subgroup. What is encouragingis that states, in collaboration with the federal government, are taking stepsto remedy some of these issues. The hope is that these continued efforts willbring more fairness into the assessment of and accountability for LEPstudents (Abedi, 2003). High Stakes / Statewide Testing The 2001 reauthorization of the Elementary and Secondary EducationAct (ESEA), also known as the No Child Left Behind (NCLB), carries testingand accountability requirements that will substantially increase studenttesting and hold all schools accountable for student performance. Thislegislation marks a major departure from the federal government‟straditional role regarding elementary and secondary education. It requiresthat states administer Reading and Mathematics tests annually in grades 3– 8 and during one year in high school starting in 2005 – 2006. Theserequirements will affect almost 25 million students each school year(National Center for Education Statistics, 2002). NCLB requires states to meet adequate yearly progress (AYP) goals toensure school accountability for student achievement on state tests. Schools
    • 44that fail to achieve AYP goals face demanding corrective actions, such asreplacement of school staff, implementation of new curriculum, extension ofthe school day or academic year, parental choice options, and, finally,complete reorganization. Today‟s widespread implementation of standards-based reform andthe federal government‟s commitment to test-based accountability ensurethat testing will remain a central issue in education for the foreseeablefuture. Test results can provide useful information about student progresstoward meeting curricular standards. But when policymakers insist onlinking test scores to high-stakes consequences for students and schools,they often overlook lessons from the long history of research (Abrams &Madaus, 2003). Current emphasis on testing as a tool of education reform continues along tradition of using tests to change pedagogical priorities and practices.In the United States, this use of testing dates back to 1845 in Boston, whenHorace Mann, then Secretary of the Massachusetts State Board ofEducation, replaced the traditional oral examination with a standardizedwritten essay test. Internationally, high-stakes testing extends as far backas the 15th century in Treviso, Italy, where teacher salaries were linked tostudent examination performance (Madaus & O‟Dwyer, 1999).Principles of Testing Programs A 1988 examination of the effects of high-stakes testing programs onteaching and learning in Europe and in the United States (Madaus, 1988)
    • 45identified seven principles that captured the intended and unintendedconsequences of such programs. Current research confirms that theseprinciples still hold true for contemporary statewide testing efforts. Principle 1: The power of tests to affect individuals, institutions,curriculum, or instruction is a perceptual phenomenon. Tests produce largeeffects if students, teachers, or administrators believe that the results areimportant. Policymakers and the public generally do believe that test scoresprovide a reliable, external, objective measure of school quality. They viewtests as symbols of order, control and attainment (Airasian, 1988). Today‟s high-stakes testing movement relies on the symbolicimportance of test scores. Forty-eight states currently require schools toprovide the public with “report cards” (Edwards, 2003). Goldhaber andHannaway (2001) found that the stigma associated with a school receiving alow grade on the state report card was a more powerful influence on Floridateachers than were the school-level sanctions imposed for poor test results. Principle 2: The more any quantitative social indicator is used forsocial decision making, the more likely it will be to distort and corrupt thesocial process it is intended to monitor. In other words, placing greatimportance on state tests can have a major influence on what takes place inthe classrooms, often resulting in an emphasis on test preparation that cancompromise the credibility or accuracy of test scores as a measure ofstudent achievement.
    • 46 We can assess whether this principle still applies today by examiningthe relationship between rising state test scores and scores on otherachievement tests. Both old and new studies of this relationship (Amrein &Berliner, 2002; Haladyna, Nolen & Haas, 1991; Klein, Hamilton, McCaffrey& Stecher, 2000; Linn, 1998) show that improvements in the state testscores do not necessarily reflect general achievement gains. We can find examples of this second principle in two recent surveys ofteachers‟ opinions. In one national study, roughly 40% of respondingteachers reported that they had found ways to raise state-mandated testscores without, in their opinion, actually improving learning (Pedulla,Abrams, Madaus, Russell, Ramos & Miao, 2003). Similarly, in a Texassurvey, 50% of the responding teachers did not agree that the rise in TexasAssessment of Academic Skills (TAAS) scores “reflected increased learningand high-quality teaching” (Hoffman, Assaf & Paris, 2001, p. 488). Principle 3: If important decisions are based on test results, thenteachers will teach to the test. Curriculum standards and tests can focusinstruction and provide administrators, teachers, and students with cleargoals. A substantial body of past data and recent research confirms that asthe stakes increase, the curriculum narrows to reflect the content sampledby the test (Jones et al., 1999; Madaus, 1991; McMillan, Myran, &Workman, 1999; Pedulla et al., 2003; Stecher, Barron, Chun & Ross, 2000). New York State, where the state department of education is requiringschools to spend more time on the NCLB-tested areas of Reading and
    • 47Mathematics, provides an example on how such pressure encouragesschools to give greater attention to tested content and decrease emphasis onnon-tested content. According to one school principal, “the art, music, andeverything else are basically out the window… something has to go”(Herszenhorn, 2003). Principle 4: In every setting where a high-stakes test operates, theexamination content eventually defines the curriculum. Pressure andsanctions associated with a state test often result in teachers using thecontent of past tests to prepare students for the new test. Several studieshave documented that an overwhelming majority of teachers feel pressure toimprove student performance on the state test. For example, 88% ofteachers surveyed in Maryland and 98% in Kentucky believed that they wereunder “undue pressure” to improve student performance (Koretz, Barron,Mitchell & Keith, 1996a, 1996b). As an outgrowth of this pressure, theamount of instructional time devoted to specific test preparation oftenincreased. Studies have found that teachers are spending a sizable amount ofinstructional time and using a variety of test-specific methods to preparestudents for their state tests (Herman & Golan, n.d.; Hoffman, Assaf, &Paris, 2001). In North Carolina, 80% of elementary teachers surveyed “spentmore than 20% of their total instructional time practicing for the end-of-grade tests” (Jones et al., 1999, p. 201). A national survey found thatteachers in high-stakes states were four times more likely than those in low-
    • 48stakes setting to report spending more than 30 hours a year on testpreparation activities, such as teaching or reviewing topics that would be onthe state test, providing students with items similar to those on the test,and using commercial test-preparation materials from previous years forpractice (Pedulla et al., 2003). Principle 5: Teachers pay attention to the form of the questions ofhigh-stakes tests (short-answer, essay, multiple-choice, and so on) andadjust their instruction accordingly. A wide variety of research confirms thattest format does influence instruction in both positive and negative ways. Studies in states that require students to formulate and providewritten responses to test questions show an increased emphasis on teachingwriting and higher-level thinking skills (Taylor, Shepard, Kinner &Rosenthal, 2003). For example, in Kentucky, 80% of teachers surveyedindicated that they had increased their instructional emphasis on problemsolving and writing as a result of the portfolio-based state test (Koretz,Barron, Mitchell, & Keith, 1996a). In several studies, teachers have reported decreases in the use ofmore time-consuming instructional strategies and lengthy enrichmentactivities (Pedulla et al., 2003). A study found that the format of the statetest may adversely affect the use of technology for instructional purposes:One-third of teachers in high-stakes states said that they were less likely touse computers to teach writing because students were required to constructhandwritten responses on the state test (Russell & Abrams).
    • 49 Principle 6: When test results are the sole or even partial arbiter offuture education or life choices, society treats test results as the major goalof schooling rather than as a useful but fallible indicator of achievement.Almost 100 years ago, a chief inspector of schools in England described thisprinciple in a way that resonates today: Whenever the outward standard ofreality (examination results) has established itself at the expense of theinward, the ease with which worth (or what passes for such) can bemeasured is ever tending to become itself the chief, if not sole, measure ofworth. And in proportion as we tend to value the results of education fortheir measurableness, so we tend to undervalue and at last ignore thoseresults which are too intrinsically valuable to be measured (Holmes, 1911). In the next five years, almost half of U.S. states will require studentsto pass a state-mandated test as a requirement for graduation (Edwards,2003). As a result, a passing score on the state test is the coin of the realmfor students, parents, teachers, and administrators. The social importanceplaced on state test scores ensures that students‟ successful performanceon the state test is the ultimate goal for schools. Local press coverage onschool pass rates and anecdotal evidence that scores on the state test mayinfluence local real estate sales show the importance of test performance asa surrogate for education quality. Principle 7: A high-stakes test transfers control over the curriculum tothe agency that sets or controls the examination. State standards-basedreform efforts leave the details and development of testing programs to state
    • 50departments of education and whomever the department contracts with toconstruct the test. This system shifts the responsibility for determiningcurricular priorities and performance standards away from local schooladministrators or classroom teachers and often results in a one-size–fits-allcurriculum and test. Falmouth, Massachusetts, provides a recent noteworthy example ofhow a high-stakes state test can override local control. Under the threat oflosing state funding and the licensure of the school principal andsuperintendent, the Falmouth School Committee reversed a decision toaward diplomas to special-needs students who failed the Massachusettsstate examination, thus shattering the hopes of a student seekingadmittance to a nonacademic culinary degree program (Myers, 2003).Accountability in Testing No one denies the importance of accountability. The relationshipbetween test scores and accountability, however, is not as simple as somepeople think. The seven principles formulated in 1988 have been acted outin state after state in the past 15 years and clearly reveal the serious flawsin the practice of using a single high-stakes measure to hold all studentsand schools accountable. Cut-off scores that place students in such performance categories asneeds improvement, basic, proficient, or advanced are arbitrary. Thesubjective methods used to categorize students into performance categoriesoften lack validity (Horn, Ramos, Blumer & Madaus, 2000). Most
    • 51policymakers and the public do not understand the psychometricunderpinnings of the tests. Issues that might seem trivial to them, such asthe assumptions made when running computer programs that producescaled scores, and even basic decisions about rounding, have significantconsequences when categorizing students. Like any measurement tool that produces a number, test scores arefallible. Yet most state laws do not consider margin of error wheninterpreting students‟ scores. Misguided executive decisions, poorlyconceived legislation, understaffing, unrealistic reporting deadlines, andunreasonable progress goals can cause numerous errors in test scores(Rhoades & Madaus, 2003). One single test can only sample knowledge and cannot give a fullpicture of what students know and can do. As an illustration, Harlow andJones‟s (2003) interviews with students showed that on the science portionof the Third International Mathematics and Science Study (TIMSS), thestudents had more knowledge about concepts than their written answershad demonstrated for more than half of the test questions. Conversely, theinterviews suggested that for one-third of the items, students lacked asound understanding of the information assessed even though they hadgiven the correct response. A fundamental principle in social science research is to always use atleast two methods when studying social science phenomena because relyingon only one method can produce misleading results. We need to enhance
    • 52state testing programs by including multiple measures of studentachievement. Measuring in a variety of ways does not mean giving studentsmultiple opportunities to take the same test, but rather incorporating othermethods of measurement or additional criteria, such as teacher judgments,when making decisions about grade promotion and graduation (Harlow &Jones, 2003). As districts, schools, and teachers respond to federal and state-basedaccountability policies, we must step back from a blind reliance on testscores. We need to acknowledge that tests, although useful, are also fallibleindicators of achievement. We also need to recognize that when test scoresare linked to high-stakes consequences, they can weaken the learningexperiences of students, transform teaching into test preparation, and taintthe test itself so that it no longer measures what it was intended to measure(Harlow & Jones, 2003).Effects of High-Stakes Testing on Student Motivation and Learning Current generation of policymakers did not invent high-stakes testing.Tests of various sorts have determined which immigrants could enter theUnited States at the turn of the 20th century, who could serve in the armedforces, who was gifted, who needed special education, and who receivedscholarships to college. But the NCLB Act of 2001 aims to make high-stakestesting more pervasive than ever before, mandating annual testing ofstudents in grades 3 – 8 in Reading and Mathematics.
    • 53 Federal legislators who overwhelmingly passed this act into lawapparently assumed that high-stakes testing would improve studentmotivation and raise student achievement. Because testing programssimilar to those required by NCLB already exist in many states, we can putthat assumption to the test. Eighteen states currently use examinations to grant or withholddiplomas: Alabama, Florida, Georgia, Indiana, Louisiana, Maryland,Minnesota, Mississippi, Nevada, New Jersey, New Mexico, New York, NorthCarolina, Ohio, South Carolina, Tennessee, Texas and Virginia. Most ofthese states also attach to their state assessments a broad range of otherconsequences for students, teachers, and schools. The experiences of thesestates can help predict how the new nationwide program of high-stakestesting will affect student achievement. Unfortunately, the evidence shows that such tests actually decreasestudent motivation and increase the proportion of students who leave schoolearly. Further, student achievement in the 18 high-stakes testing states hasnot improved on a range of measures, such as the National Assessment ofEducational Progress (NAEP), despite higher scores on the state‟s ownassessments. (Amrein & Berliner, 2003). High-stakes testing assumes that rewards and consequences attachedto rigorous tests will “motivate the unmotivated” to learn (Orfield &Kornhaber, 2001). The “unmotivated” are usually identified as low socio-economic students in urban schools, often African Americans and Latinos.
    • 54Researchers have found that when rewards and sanctions are attached toperformance on tests, students become less intrinsically motivated to learnand less likely to engage in critical thinking. In addition, they have foundthat high-stakes testing cause teachers to take greater control of thelearning experiences of their students, denying their students opportunitiesto direct their own learning. When the stakes get high, teachers no longerencourage students to explore the concepts and subjects that interest them.Attaching stakes to tests apparently obstruct students‟ path to becominglifelong, self-directed learners and alienates students from their ownlearning experiences in school (Sheldon & Biddle, 1998). Dropout rates are climbing throughout the United States and manyresearchers hold high-stakes testing at least partly to blame (Rothstein,2002). Some researchers found that dropout rates were 4 to 6 percenthigher in schools with high school graduation examinations. Another studyreported that students in the bottom quintile in states with high-stakestesting were 25% more likely to drop out of high school than were theirpeers in states without high-stakes testing (Jacob, 2001). Researchers in yetanother study found that failing these tests significantly increased thelikelihood that even students with better academic records would drop out(FairTest & Massachusetts CARE, 2000). More and more teenagers are exiting formal schooling early to earn aGeneral Educational Development (GED) credential (Murnane, Willett, &Tyler, 2000). Although young people who have earned such alternative
    • 55degrees do not technically count in dropout statistics, many of themundoubtedly left school because of their concerns about passing rigorousgraduation tests. Students who repeat a grade are significantly more likely to drop outof school (Goldschmidt & Wang, 1999). In states where promotion to thenext grade hinges on passing the state exams, high-stakes testing policiescontribute to higher dropout rates in the long run. Even before they actuallytake the test, struggling students are more likely to be retained in grade ifthey attend schools in high-stakes testing environments. By holding low-achieving students back, schools ensure that these students have more ofthe knowledge necessary to perform well on high-stakes testing the nextyear and keep low-performing students‟ test scores out of the composite testperformance in the grades in which high-stakes testing matter. In Texas, students from racial minorities and low socio-economicbackgrounds are being retained in Grade 9 at very high rates before takingthe Texas Assessment of Academic Skills (TAAS) in Grade 10. Many teachersretain students if they doubt their potential to pass TAAS the following year.McNeil (2000) estimated that half of all minority students enrolled in Texashigh schools are technically enrolled as freshmen. Although some of themare 9th graders for the first time, thousands of others have been retained inthe 9th grade once or even twice. Other researchers (Haney, 2000, 2001;Klein, Hamilton, McCaffey & Stecher, 2000; Yardley, 2000) have verified hernumbers. In 1998, one in every four African American and Latino 9 th
    • 56graders in Texas was retained (Fisher, 2000). After these students areretained, thousands of them drop out of school. Common problems of high-stakes testing programs are quite likely toaffect the breadth and depth of student learning. If schools narrow thecurriculum they teach; make heavy use of drill activities tied to the statetest; cheat by over-identifying language minority and special educationstudents and then keeping these students from taking these tests; retainpoorly performing students in grade; and encourage those who are at leastlikely to pass the state‟s test to drop out, then scores on state tests willalmost certainly go up. But have students really learned any more than theydid before high-stakes testing policies were instituted (Fisher, 2000)?Other Considerations of Assessment and Testing Although NCLB now requires all students to be accounted for in anystate‟s assessment system, this has not always been the case (Anderson,2004). In the past, groups of students such as English language learners orstudents in Special Education were systematically excluded from large scaleassessments (State, 1999), or their scores were not reported (Thurlow,Neilson, Tellucksingh, & Ysseldyke, 2000). In the 2002-2003 school year, the Texas Assessment of Knowledgeand Skills (TAKS) replaced the Texas Assessment of Academic Skills (TAAS)as the primary statewide assessment program. TAKS is designed bylegislative mandate to be more comprehensive than its predecessors andencompasses more of the state-mandated curriculum, the Texas Essential
    • 57Knowledge and Skills (TEKS), at more grade levels than TAAS did. The highschool level assessments, administered at Grades 9, 10 and 11, aregrounded in the high school TEKS curriculum. By law, students for whomTAKS is the graduation testing requirement must pass exit level tests in fourcontent areas – English Language Arts, Mathematics, Science, and SocialStudies – in order to graduate from a Texas public high school (TechnicalDigest, 2003-2004). In Texas, there is evidence that the numbers of black and Hispanicstudents in Special Education rose between 1994-1998 while the stateimplemented its statewide testing program which excluded some SpecialEducation students from the reported scores (Haney, 2000). It is interestingto note that while the numbers of African-American and Hispanic studentsare over-represented in Special Education, about eight to nine percent ofEnglish language learners are identified as receiving Special Educationservices in the US (D‟Emilio, 2003, June; Zehler, Fleischman, Hopstock,Pendzick, & Stepherson, 2003). Labeling schools can have an impact on teacher and student morale(Anderson, 2004). Certainly, poor test scores or poorly explainedassessment systems can result in decreased student motivation (Lane &Stone, 2002). Teachers have also reported that the high-stakes nature ofsome assessments can have a negative impact on student morale (Flores &Clark, 2003). Although some teachers have reported that their English
    • 58language learners can reach the high standards set for them, they may needmore time than other students (Hood, 2003). For English-language learners, the additional requirements of an exitexamination could increase dropout rates (Anderson, 2004). Hispanicstudents, many of whom are English-language learners, have higherdropout rates than the population as a whole (Barro & Kolstad, 1987;Kaufman, Alt, & Chapman, 2001). In another study, teachers reported that increasing emphasis on testscores cause them to dislike their jobs (Hinde, 2003). In a study examiningthe discussion and journal entries from teachers, Flores and Clark (2003)found that teachers were not against accountability and viewed it as distinctfrom statewide testing, but also thought that an over-emphasis on testingresulted in unbalanced curriculum and inappropriate instructionaldecisions. In order for teachers to make specific changes to instruction, theassessments needs to be clear as to what skills are being assessed (Popham,2003). Some teachers may react to low test scores of English languagelearners by teaching to the test while others may ignore the impact of thetest scores all together (Alderson & Wall, 1993). Testing cannot be divorced from socio-cultural, economic, andpsychological issues (Solano-Flores & Trumbull, 2003). ELLs, for instance,may not score any differently in an assessment even when allowed to use adictionary (Albus, Thurlow, Liu & Bielinski, 2005). This is furthercomplicated by the theory forwarded by Wang and Koda (2005) that ELLs as
    • 59a group may have diverse styles in developing English Language proficiency.Therefore modifications are adapted to teach ELLs the academic content of alesson, and at the same time support the acquisition of a new language(Major, Fitzmaurice, Bunta, & Balasubramanian, 2005). A study of washback from a test in Hong Kong demonstrated thatchange in the assessment could change the ways in which teachers andstudents interacted (Cheng, 1999). School Administrators and teachers, aswell as students, need to be motivated to change the way learning takesplace and also be invested in demonstrating achievement on theassessments in order for washback to instruction to take place and besuccessful (Lane & Stone, 2002). Related Studies In a study on “Intended and Unintended Consequences of StatewideTesting for ESL Curriculum and Instruction”, Anderson (2004) examinedwhat positive or negative impact assessment systems have on thecurriculum and instruction of English language learners in one Midwesternschool district. The researcher used focus groups and interviews to obtainviews of educators on the consequences of statewide testing for ELLs.Positive consequences that were identified included more teachercollaboration, changes in curriculum and instruction, better alignmentbetween ESL and content area curricula and more focus on reading andwriting. Negative consequences included student and teacher frustration,more teaching to the test occurring, and a narrowed curriculum. Educators
    • 60in the study also identified problems with the accountability system andmade recommendations for how it could be improved (Anderson, 2004). Another study on “Inclusion of Students with Limited EnglishProficiency in National Assessment of Educational Progress (NAEP):Classification and Measurement Issues” conducted by Abedi (2004) reportedthe major concerns over classification and measurements for students withlimited English proficiency (LEP). Issues included the poor operationaldefinition of English proficiency construct and validity concerns on theexisting language proficiency tests. The study discussed issues concerningthe classification of ELLs and elaborated on factors that impact decisions toinclude ELLs in NAEP assessments. With funding through a competitivebidding process authorized under the NCLB section on EnhancedAssessment Instruments, there are national efforts underway to developEnglish proficiency tests that can be used to provide valid measures ofstudents‟ level of English proficiency (Abedi, 2004). Wall (2000) made a microethnographic case study entitled “A CaseStudy of Secondary School Efforts Toward English Language LearnerSuccess in a Standards-Based Reform System.” This study was designed todescribe and interpret the site-based decision-making process of acollaborative study group of high school educators as they focused on theappropriate participation of ELLs in a district wide, standards-based, reforminitiative. The research question which guided the study was: From whatperspectives and with what outcomes does a collaborative group of site-
    • 61based, high school educators deliberate the participation of ELLs in astandards-based reform system which mandates high stakes assessments?Three themes emerged from the study: (a) personal discovery, (b) informedaction, and (c) instructional advocacy. These themes suggested phases ofsociolinguistic accommodation through which educators progress in theirreform-based deliberations regarding appropriate approaches to supportELLs in a high-stakes assessment system (Wall, 2000). This study on the impact of high-stakes testing on ELLs in majorurban high schools in Texas showed quantitatively how the percentage ofELLs enrolled in a school affects the school‟s performance in the State‟sassessment. Qualitatively, it gathered the input and feedback of educatorson the different concerns included in the study: (a) purpose of TAKS, (b)changes caused by TAKS, (c) consequences of TAKS, (d) recommendations toimprove TAKS, and (e) needs of ELLs. Summary As stated in chapter I, the purpose of this study was to determine theimpact of high-stakes testing on ELLs. This was shown in both thequantitative and qualitative dimensions of the study. Data obtained fromTEA were used to determine whether there is a relationship between thepercentage of English language learners enrolled in a school and thepercentage of all students passing the 10th grade TAKS tests in the coreareas of English Language Arts and Mathematics given in 2003, 2004, 2005and 2006. To support the quantitative aspect, this study explored what
    • 62certified ESL teachers, non-certified ESL teachers who teach ELLs,administrators, and district ESL personnel viewed as the impact that highstakes standardized assessments have on ELLs, ESL curriculum andinstruction, and what they observed as actually occurring. The mandates and key elements of the NCLB were geared towardsimproving the achievement of students in the different public schools of theUnited States. The measure of adherence was channeled through the AYPthat the different schools and districts of the different states monitor andreport. High-stakes testing became the measuring stick that gauged theachievement of students in the different core subject areas. Issues andconcerns were centered on the ELLs regarding the different moves andaccommodations given to this special subgroup of learners. Feedbackregarding the issues and concerns of the different studies and researchesincluded both positive and negative dimensions.
    • CHAPTER III METHODOLOGY Introduction Standardized testing and assessments have become necessaryfacets of American education. Consequently, accountability testing iscurrently implemented in practically every state in the U.S. Since thepurpose of this increased level of accountability is to ensure that allstudents are receiving a quality, standards-based education, it isimportant to document the consequences of the system to ensure thatthe intended reforms are taking place. One of the goals of theaccountability system should be to document any negative opposingimpact that could occur so that interventions can be developed so thatthese consequences can be minimized. The purpose of this study was to determine the impact of high-stakes testing on ELLs. This is shown in both the quantitative andqualitative dimensions of the study. Data obtained from TEA were usedto determine whether there is a relationship between the percentage ofEnglish language learners enrolled in a school and the percentage of allstudents passing the 10th grade TAKS tests in the core areas of EnglishLanguage Arts and Mathematics given in 2003, 2004, 2005 and 2006. Tosupport the quantitative aspect, this study explored what certified ESLteachers, non-certified ESL teachers who teach ELLs, administrators,and district ESL personnel viewed as the impact that high stakes 63
    • 64standardized assessments have on ELLs, ESL curriculum andinstruction, and what they observed as actually occurring. The study also addressed concerns regarding the validity ofstudent evaluations, and the common inferences made about studentperformance in these assessments. There is a need to know how publicschools that have diverse student attributes can be held accountable onthe basis of one uniform and universal standard. Since the standardizedassessments are given in English, schools with predominantly Hispanicpopulations may already be at a disadvantage through no fault of theirown. What needs to be examined is whether standardized assessmentsfacts are free from linguistic and cultural bias as viewed by teachers ofELLs. Research QuestionsQuantitative Is there a relationship between the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005 and 2006? Null HypothesesHO1: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005 and 2006.
    • 65HO2: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005 and 2006.Qualitative The major question answered by this study was: What are theanticipated and observed consequences of statewide testing specifically,TAKS, on ELLs, ESL curriculum and instruction as viewed by certifiedESL teachers, non-certified ESL teachers who teach ELLs, schooladministrators, and district ESL personnel? This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
    • 66 Research Methods Both descriptive and comparative research techniques wereemployed in the explanatory design of the mixed methods study.Fraenkel and Wallen (2003) stated that Creswell describes the two typesof mixed methods. 1. In a triangulation design, the researcher simultaneously collects both quantitative and qualitative data, compares results, and then uses those findings to see whether they validate each other (p. 443). 2. In an explanatory design, the researcher first collects and analyzes quantitative data, and then obtains qualitative data to follow up and refine the quantitative findings (p. 443). For this study, the explanatory design was used. Quantitative datafor this research were gathered through TEA to determine if arelationship existed between the percentage of English language learnersenrolled in a school and the percentage of all students passing the 10thgrade TAKS tests in the core areas of English Language Arts andMathematics given in 2003, 2004, 2005 and 2006. Qualitative data wereobtained through the on-line, open-ended questionnaire and individualand focus group interviews about the varied ways in which standardizedassessments impacted ELLs. For the qualitative research component, the study used the cross-sectional, open-ended questionnaire. A cross-sectional, open-ended
    • 67questionnaire collects information from a sample that has been drawnfrom a predetermined population. Furthermore, the information iscollected at just one point in time, although the time it takes to collectthe data may take anywhere from a day to a few weeks or more (Fraenkel& Wallen, 2003). The study also utilized descriptive research methods. Isaac andMichael (1995, p. 46) describes this type of research as: “to describesystematically a situation or area of interest factually or accurately.” Asurvey study also falls under the classification of descriptive research.Van Dalen (1979) lists the purpose of survey studies: 1. To collect detailed factual information that describes existing phenomena. 2. To identify problems or justify current conditions and practices. 3. To make comparisons and evaluations. 4. To determine what others are doing with similar problems or situations and benefit from their experience in making future plans and decisions. Research Design Since the study utilized the explanatory design of the mixedmethods, the investigator first gathered quantitative data from TexasEducation Agency (TEA) regarding the major urban high schools inTexas. TEA records personnel assisted in accessing and retrieving datafrom the TEA website. Qualitative data were obtained through the on-
    • 68line, open-ended questionnaire and individual and focus groupinterviews; views and opinions of the respondents were gathered andcollated to validate and support the quantitative data.Quantitative Data From the Texas Education Agency, the following data regarding theurban high schools were gathered: the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005 and 2006.Qualitative Data Qualitative data were obtained using an on-line, open-endedquestionnaire given to principals, assistant principals, ESL districtpersonnel, ESL certified teachers and non-ESL certified teachers whowere purposively sampled for the study and through the individual andfocus group interviews using open-ended questions about the variedways in which standardized assessments impact ELLs. Pilot Study Two Houston Independent School District schools, not included inthe main study were selected for the pilot study. Quantitative data wereobtained regarding the schools’ percentage of English language learnersenrolled in a school and the percentage of all their 10th grade studentspassing TAKS in the two core areas of English Language Arts and
    • 69Mathematics. This was for the four school years starting with the firstschool year 2002 – 2003, when TAKS was administered. During the pilot study the on-line questionnaire underwent pre-testing with three basic considerations: (1) administered the pre-testunder conditions comparable to those anticipated in the final study; (2)analyzed the results to assess the effectiveness of the trial questionnaireto yield the information desired; and (3) made appropriate additions,deletions, and modifications to the questionnaire (Isaac & Michael,1995). Qualitative data resulting from an on-line open-endedquestionnaire on the six different concerns listed below were tabulatedcombining the results from the two schools. Results were categorizedusing the NVivo software package but the categories were modified basedon the expert opinion of the respondents belonging to the focus groups.The frequencies for the responses by the different respondents (teachers,school administrators and district ESL personnel) pertaining to thedifferent categories were tallied and percentages were computed. Listingof categories was based on the total frequencies; those categoriesidentified most by the respondents were listed first followed by those withlower frequencies. The different concerns included the following: (1)Purpose of TAKS; (2) Consequences of TAKS; (3) Problems Related toTAKS; (4) Changes Caused by TAKS; (5) Recommendations to ImproveTAKS; and (6) Needs of ELLs.
    • 70 Results of the focus group and one-on-one interviews werevalidated against the results of the on-line questionnaire and providedexplanation or support for the answers given. The categories for thedifferent responses were affirmed or modified by the focus groups. Population and SamplesQuantitative Data The TEA provided the data on the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005 and 2006. The researcherused purposive sampling in selecting schools for this study. Purposivesampling is based on the assumptions that the investigator wants todiscover, understand, and gain insight and therefore must select asample from which the most can be learned (Merriam, 1998). While not a random sampling of high schools, the sample is notintended to create results that can be generalized to all major urban highschools in the U.S. A purposive sampling was used in order to provide arepresentative sample of the major urban high schools in Texas in orderto gain in-depth insight into what impact might be occurring. The impactthat might emerge from this study might occur in other high schools, butit is important to take into account the characteristics of the highschools as well as the assessment system in the state in order to
    • 71extrapolate from the findings and make comparisons with othersituations (Patton, 1990).Qualitative Data The on-line, open-ended questionnaire was given to the principals,assistant principals, certified ESL teachers and non-ESL certifiedteachers handling ELLs of the selected schools and to the districtpersonnel: Total 1) ESL Teachers 30 2) Non-Certified ESL Teachers 30 3) Principals 10 4) Assistant Principals 20 5) District ESL Personnel 8 Total 98 The different focus groups consisted of ESL certified and non-ESLcertified teachers handling ELLs. One-on-one interviews involved theselected principals and the selected district ESL personnel. The sameschools and district personnel who answered the on-line questionnairewere included in the focus groups and one-on-one interviews. Selectionof the participants in the focus group interviews utilized the snowballingtechnique. Participants will identify others whose input or experience willalso be valuable to the study (Krathwohl, 1993).
    • 72 Since the researcher has the obligation to respect and protect therights and wishes of the research participants, the following actions weredone: (1) the researcher protected anonymity of the participants by usingcomputer-given codes for the responses; and (2) the researcher informedthe participants about the purpose of the survey. The security of the raw data gathered through the records sectionsof TEA and the selected schools, responses to the on-line questionnaireand the transcripts of the interviews was assured in order to protect theanonymity of the participants and to uphold the trustworthiness of thestudy. The above concerns regarding trustworthiness and confidentialityof data or information were shared with the participants when theresearcher contacted them through e-mail, telephone, mail, or in person. Instrumentation Quantitative data were accessed and retrieved from the TEAwebsite regarding the major urban high schools in Texas. Data wereorganized for computations utilizing the SPSS software package, Version14.0. The on-line, open-ended questionnaire provided one of the basesfor the qualitative data. The triangulation method included categorizingthe responses to the online, open-ended questionnaire into emergentthemes, interviewing the focus groups of teachers and assistant
    • 73principals and one-on-one interviews with the principals and district ESLpersonnel. According to Fraenkel and Wallen (2003) personal interview isprobably one of the most effective ways there is to enlist the cooperationof respondents in a survey; rapport can be established, questions can beclarified, unclear or incomplete answers can be followed up and so on. Patton (1990) expounds that the purpose of interviews is to gainaccess to those areas of the participants’ experiences or thought whichcannot be observed. Consequently, interviews will play a significant rolein data collection, a role which generally cannot be duplicated by othermeans (Dexter, 1970). Fraenkel and Wallen (2003) cite the following advantages of open-ended questions in survey research (a) allows more freedom of response;(b) easier to construct; and (c) permits follow-up by interviewer. But thereare also disadvantages: (a) responses tend to be inconsistent in lengthand content across respondents; (b) both questions and responses maybe subject to misinterpretation; and (c) responses are harder to tabulateand synthesize. However, these disadvantages can be minimized throughthe use of the NVivo software package, expert help from the focus groupsin classifying categories, follow-up interviews with the focus groups andone-on-one interviews.
    • 74 Validity and Reliability For validity and reliability, the following expert opinions wereconsidered. “Validity, I mean truth: interpreted on the extent to which anaccount accurately represents the social phenomena to which it refers”(Hammersley, 1990, p. 57). “Reliability refers to the degree of consistencywith which instances are assigned to the same category by differentobservers or by the same observer on different occasions” (Hammersley,1990, p. 67). The triangulation method involving the analysis of thequalitative data, collation of data from the on-line questionnaire andinterviews assured the validity and reliability of the survey questions andthe explanatory design of the mixed methods study. For the quantitative dimension of the study, validity and reliabilitymeasurements were derived from the TAKS report prepared by TEA.Validity is a process of collecting evidence to support inferences madefrom scoring results of an assessment. In the case of TAKS, test resultsare used to make inferences about the students’ knowledge andunderstanding of the Texas Essential Knowledge and Skills (TEKS). Testreliability indicates the consistency of measurement. TAKS testreliabilities are based on internal consistency measures, in particular onthe Kuder-Richardson Formula 20 (KR-20) for tests involvingdichotomously scored (multiple choice) items and on the stratifiedcoefficient alpha for tests involving a mixture of dichotomous andpolytomous (essay-prompt and short answer) items.
    • 75 In order to build trustworthiness in the qualitative aspect of thestudy, four different criteria were considered to meet this need:(1)credibility, which aims to produce findings that are believable andconvincing; (2) transferability, which attempts to apply findings in onesetting to other contextually similar settings; (3) dependability, whichaddresses the question concerning which findings are consistent withthose of other similar investigations; and (4) confirmability, whichensures that both the process and the product are auditable (Isaac &Michael, 1995). Research ProceduresQuantitative After appropriate permissions for data gathering were obtained,records personnel of TEA were contacted and arrangements made as toprocess and assistance regarding acquisition of data for the study. Thedry-run or pilot study with the two HISD schools facilitated the aboveprocess.Qualitative The questions in a survey, and the way they are asked, are ofcrucial importance (Fraenkel & Wallen, 2003). The authors refer to FloydFowler who points out that there are four practical standards that allsurvey questions should meet: 1. Is this a question that can be asked exactly the way it is written?
    • 76 2. Is this a question that will mean the same thing to everyone? 3. Is this a question that people can answer? 4. Is this a question that people will be willing to answer, given the data collection procedures? (Fowler, 1984). After the questionnaire was refined based on the suggestions of thefocus groups during the pilot study, the questionnaire was placed on-lineto respondents of the study. Prior to this, the researcher contacted therespondents in person, by phone, by email or mail. Furthermore, theresearcher arranged dates with the different schools and districts for thefocus group and one-on-one interviews. Data Collection and RecordingQuantitative The data for the major urban high schools regarding thepercentage of ELLs enrolled in a school and the percentage of all Grade10 students passing the TAKS tests in English Language Arts andMathematics during the four years when TAKS was administered,starting school year 2002 – 2003 were obtained from the TEA website.Qualitative An on-line, open-ended questionnaire was answered by theprincipals, assistant principals, ESL teachers, and non-certified teachershandling ELLs of the selected major urban high schools in Texas. DistrictESL personnel were also requested to answer the same questionnaire.The focus groups offered expert opinions regarding the categories to use
    • 77in classifying the responses to the questionnaire. Further clarificationwas requested from the principals and the district ESL personnel duringthe one-on-one interviews. Results of the questionnaire were placed in categories suggested bythe focus groups after initial classification was done through the NVivosoftware system. Transcripts of the interviews and focus groups wereentered into the NVivo software system (version 7.0) and coded accordingto themes that emerged in the data. NVivo provides a sophisticated wayof electronically organizing interview transcripts for analysis andclassification into themes and allowed the researcher to work with a largeamount of transcript data. The themes that emerged from the data werecompiled and compared between high schools. While NVivo was avaluable sorting tool that allowed the researcher to code, sort, and recalldata in different ways, the researcher developed and created codes forthe responses gathered. The analysis was done by the researcher usingNVivo’s capabilities to sort out the complexities of the rich data from theinterviews and focus groups. A program such as NVivo can help theresearcher ensure that the qualitative data were well-organized(Weitzman, 2000). One of the strengths of collecting qualitative data is the richness ofthe information that can be collected and which can capture a theme in amore complete way than the researcher may be able to summarize. Thisevidence directly from the data was used to show a clear connection
    • 78between the data and the identified themes (Marshall, 1990). The richdescription of the themes from the participants’ own words also aids inverifying that the themes identified are those that the participantsactually voiced (Creswell, 1998; Krueger & Casey, 2000). The researcher triangulated quantitative data analysis, qualitativedata analysis, and interviews in order to strengthen the credibility of thesurvey study. By combining multiple observers, theories, methods, anddata sources, researchers can hope to overcome the intrinsic bias thatcomes from single-method, single-observer and single-theory studies(Denzin, 1970). With the mix of analyses, the author has better tools todiscuss the impact of statewide testing on ELLs, ESL curriculum andinstruction. Data AnalysisQuantitative Descriptive statistics and analyses were performed to test eachvariable. After the data were examined and properly inputted, the nextstep was to compute for Pearson r correlation coefficients using the SPSSstatistical package and test for statistical relationship at p < 0.05. Forother analyses, the predictor variable is the percentage of ELLs enrolledin a school and the outcome variables were the percentages of allstudents passing the Grade 10 TAKS tests in English Language Arts andMathematics. For each of the years under study, two separate Pearson rcorrelations were computed; the percentage of ELLs enrolled in a school
    • 79was compared with the English Language Arts results to determine ifthey have significant relationship and the other comparison was with theMathematics results. The SPSS computations showed three differentresults in tabular form: (1) the means and the standard deviations of thepercentage of ELLs enrolled in a school and the percentages of allstudents passing the TAKS tests in English Language Arts andMathematics during the four years under study; (2) Pearson r correlationcoefficients to determine if there was significant relationship between thepercentage of ELLs enrolled in a school and the percentages of allstudents passing the English Language Arts and Mathematics TAKStests given in 2003, 2004, 2005 and 2006; and, (3) regression analysisusing the percentage of ELLs enrolled in a school as the predictorvariable to predict the percentage of students passing in the 10th gradeTAKS tests in the core areas of English Language Arts and Mathematics.Qualitative The information for the qualitative portion of the study includedthe emergent themes shown as categories in the frequency distributiontable. The frequency distribution is a table in which all score units arelisted in one column and the number of individuals receiving each scoreappears as frequencies in the second column (Isaac and Michael, 1995).Frequencies were tallied and percentages were computed. Categorieswith higher percentages were listed first followed by those with lowerpercentages. An overview preceded each table giving the emergent
    • 80themes mostly cited by the respondents. Anecdotal records followed thetables - these are the views and opinions of the respondents regardingthe different concerns included in the study. Summary In this study the researcher considered the aspects of proceduralconsistency, neutrality of findings, and truth value to assure the study oftrustworthiness. “Valid inquiry in any sphere… must demonstrate itstruth value, provide the basis for applying it, and allow for externaljudgments to be made about the consistency of its procedures and theneutrality of its findings or decisions” (Erlandson, 1993). Quantitative data that were sourced as aggregate data from theTEA website included the percentage of English language learnersenrolled in a school and the percentage of all students passing the 10thgrade TAKS tests in the core areas of English Language Arts andMathematics given in 2003, 2004, 2005 and 2006. Qualitative data were collated from the responses of selectedrespondents to the on-line questionnaire regarding the anticipated andobserved consequences of the statewide testing, specifically TAKS, onESL curriculum and instruction as viewed by ESL teachers, schooladministrators and district ESL personnel. Interviews were conductedwith the focus groups and one-on-one interviews involved the principalsand district ESL personnel.
    • 81 Presentation of data included: (a) the quantitative data analysis onthe correlation between the percentage of ELLs enrolled in a school andthe percentage of all students passing the 10th grade TAKS in EnglishLanguage Arts and Mathematics and the regression analysis using thepercentage of ELLs enrolled in a school as predictor variable and thepercentage of all students passing the 10th grade TAKS in ELA andMathematics as outcome variables; (b) qualitative data analysisclassifying responses to the on-line, open-ended questionnaire asdifferent emergent themes; and (c) anecdotal records from the interviewswith the different focus groups, principals, and district ESL personnel. The relationship between the percentage of ELLs enrolled in aschool and the percentage of all students passing the 10th grade TAKS ineach of the core areas of English Language Arts and Mathematics wasdetermined using the SPSS program for Pearson r correlation. Theregression analysis resulted to linear regression equations predicted thepercentage of all students passing in 10th grade TAKS in EnglishLanguage Arts and Mathematics using the percentage of ELLs enrolled ina school as predictor variable. Emergent themes were categorized through the NVivo softwarepackage and suggestions of the focus groups. The anecdotal recordsexpressed the views and opinions of the respondents regarding thefollowing and focused on the ELLs: (a) the purpose of the statewide, highstakes TAKS; (b) intended consequences of TAKS; (c) problems related to
    • 82TAKS; (d) changes caused by TAKS; (e) recommendations to improveperformance in TAKS; and, (f) the needs of ELLs.
    • CHAPTER IV ANALYSIS OF DATA The purpose of this study was to determine the impact of high-stakes testing on English Language Learners (ELLs). This was shown inboth the quantitative and qualitative dimensions of the study. Bothquantitative and qualitative dimensions of the study provided the statusof high-stakes testing as it affected ELLs and how it influenced efforts inschools to improve performance of students, particularly ELLs. Dataobtained from Texas Education Agency (TEA) were used to determinewhether there was a relationship between the percentage of ELLsenrolled in a school and the percentage of all students passing the 10thgrade Texas Assessment of Knowledge and Skills (TAKS) tests in the coreareas of English Language Arts and Mathematics given in 2003, 2004,2005, and 2006. To support the qualitative aspect, this study exploredwhat certified English as a Second Language( ESL) teachers, non-certified ESL teachers who teach ELLs, administrators, and district ESLpersonnel viewed as the impact that high stakes standardizedassessments had on ELLs, ESL curriculum and instruction, and whatthey observed as actually occurring. With the No Child Left Behind (NCLB) Act of 2001, studentachievement has been placed in the forefront of quality education.Aligned to this effort is statewide testing aimed at assessing students‟performance and status of the school. In this chapter, the relationships 83
    • 84between the percentage of English language learners enrolled in a schooland the percentage of all students passing the 10th grade TAKS tests inthe core areas of English Language Arts and Mathematics given in 2003,2004, 2005, and 2006 are shown. The results of the Statistical Packagefor the Social Sciences (SPSS) computations show the: (1) means andthe standard deviations of the percentage of ELLs enrolled in a schooland the percentages of all students passing the 10th grade TAKS tests inEnglish Language Arts and Mathematics during the four years understudy; (2) Pearson r correlation coefficients between the percentage ofELLs enrolled in a school and the percentages of all 10th grade studentspassing the English Language Arts and Mathematics TAKS tests given in2003, 2004, 2005, and 2006; and, (3) regression analysis using thepercentage of ELLs as the predictor variable. Following this quantitative dimension of the study is the qualitativeinformation regarding TAKS and its effects on the ELLs - its intendedpurpose and consequences, problems encountered related to thisstatewide testing, changes that occurred due to TAKS andrecommendations to improve the performance of ELLs in general andspecifically on this high-stakes test. The tabulated results of the open-ended, on-line questionnaire and the supporting explanation gatheredthrough the individual and focus group interviews comprised thequalitative portion of this study. ESL district personnel, principals,
    • 85assistant principals, ESL-certified teachers and non-ESL certifiedteachers handling ELLs provided the needed information. Schools with ELLs are asked to account for their performance inline with NCLB. School and district administrators may get feedbackboth from the numerical results of high-stakes testing and from inputs ofteachers and other personnel responsible for handling ELLs. Since thepresence of ELLs in schools is a reality administrators have to face, itmay be beneficial to be aware of the views and opinions of concernedpersonnel in the school system regarding interventions that may improvethe curriculum and/or instruction of ELLs. Findings For the quantitative portion of the study, data for the 173 urbanhigh schools were obtained from Texas Education Agency; however, forthe year 2005, data available were only for 155 high schools. The datagathered were collated and the SPSS software package was utilized forthe needed computations. The following research question was the focusof the quantitative dimension of the study:Research Question - Quantitative Is there a relationship between the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005, and 2006?
    • 86 Two null hypotheses were formulated to answer the abovequestion:HO1: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in English Language Arts given in 2003, 2004, 2005, and 2006.HO2: There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in Mathematics given in 2003, 2004, 2005, and 2006. The results are presented in the following order: (a) the descriptivestatistics (mean, standard deviation, number of cases); (b) the Pearson rcorrelation coefficients; and, (c) regression analysis showing the linearequations which can be used to predict the outcomes in the 10 th gradeTAKS tests in English Language Arts and Mathematics. Tables 4.1.1 to 4.1.4 show that the average percentage of ELLs inthe urban high schools of Texas during the four years under studystarting from 2003, ranged from 6.94% to 8.30 %; the average percentageof ELLs enrolled in a school was 7.79%.
    • 87Table 4.1.1Means and Standard Deviations of ELLs Enrolled in School and AllStudents Passing the 2003 10th Grade TAKS for English Language Artsand Mathematics. Mean Std. Deviation NELLs Enrolled 08.31 08.98 165In SchoolAll StudentsPassing ELA 62.87 18.11 151All StudentsPassingMathematics 61.85 20.84 151Table 4.1.2Means and Standard Deviations of ELLs Enrolled in School and AllStudents Passing the 2004 10th Grade TAKS for English Language Artsand Mathematics. Mean Std. Deviation NELLs Enrolled 08.30 09.64 147In SchoolAll StudentsPassing ELA 68.28 16.24 130All StudentsPassingMathematics 53.57 22.44 132
    • 88Table 4.1.3Means and Standard Deviations of ELLs Enrolled in School and AllStudents Passing the 2005 10th Grade TAKS for English Language Artsand Mathematics. Mean Std. Deviation NELLs Enrolled 7.62 10.80 147In SchoolAll StudentsPassing ELA 59.39 20.13 135All StudentsPassingMathematics 47.68 23.35 134Table 4.1.4Means and Standard Deviations of ELLs Enrolled in School and AllStudents Passing the 2006 10th Grade TAKS for English Language Artsand Mathematics. Mean Std. Deviation NELLs Enrolled 6.94 9.94 167In SchoolAll StudentsPassing ELA 78.05 13.68 155All StudentsPassingMathematics 50.13 22.40 152
    • 89 Tables 4.1.1 through 4.1.4 reflect the descriptive statistics showingthe percent of all students passing in English Language Arts were:62.87% in 2003, 68.28% in 2004, 59.39% in 2005 and 78.05% in 2006.The descriptive statistics showing the results of all students passing inMathematics were: 61.85% in 2003, 53.57% in 2004, 47.68% in 2005and 50.13% in 2006. Based on the above tables, Tables 4.1.1.1 and 4.1.1.2 compare theresults of 10th grade TAKS in English Language Arts and Mathematics.Table 4.1.1.1Comparison of Results in 10th Grade English Language Arts TAKS Percent Means-All Means-All ELLs Students Students Enrolled Passing Passing Percent in Current Previous Difference Increase Year School Year Year (+/-) (Decrease) 2003 8.30 62.87 -- -- -- 2004 8.29 68.28 62.87 +5.41 08.61 2005 7.62 59.39 68.28 -8.89 -13.02 2006 6.94 78.05 59.39 +18.66 031.42
    • 90Table 4.1.1.2Comparison of Results in 10th Grade Mathematics TAKS Percent Means-All Means- All ELLs Students Students Enrolled Passing Passing Percent in Current Previous Difference Increase Year School Year Year (+/-) (Decrease) 2003 8.30 61.85 -- -- -- 2004 8.29 53.57 61.85 -8.28 -13.39 2005 7.62 47.68 53.57 -5.89 -10.99 2006 6.94 50.13 47.68 +2.45 05.14 Tables 4.2.1 to 4.2.4 show the obtained Pearson r correlationcoefficients for the correlations between percent of ELLs enrolled in aschool and the percent of all students passing in 10th grade TAKS test inEnglish Language Arts and Mathematics. The obtained Pearson rcorrelation coefficients between the percentage of ELLs enrolled in aschool and the percentage of all students passing in 10 th grade TAKSwere: -0.349 in 2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006for English Language Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in2005 and -0.356 in 2006 for Mathematics. All the computed Pearson rcorrelation coefficients were significant at the 0.05 level, two-tailed. Wetherefore reject the null hypotheses. All the obtained Pearson rcorrelation coefficients were negative for both content areas; this inverserelationship indicated that as the percentage of ELLs increased, the
    • 91percentage of all 10th grade students passing in each of the core areas ofEnglish Language Arts and Mathematics decreased.Table 4.2.1Pearson Correlations: 2003 10th Grade TAKS for English Language Artsand Mathematics. Percent of All Percent of All Students StudentsPearson Correlation Passing ELA Passing MathPercent of ELLs Enrolled -.349(**) -.293(**)Sig. (2-tailed) .00000 .00000** Correlation is significant at the 0.05 level (2-tailed).Table 4.2.2Pearson Correlations: 2004 10th Grade TAKS for English Language Artsand Mathematics. Percent of All Percent of All Students StudentsPearson Correlation Passing ELA Passing MathPercent of ELLs Enrolled -.392(**) -.351(**)Sig. (2-tailed) .00000 .00000** Correlation is significant at the 0.05 level (2-tailed).
    • 92Table 4.2.3Pearson Correlations: 2005 10th Grade TAKS for English Language Artsand Mathematics. Percent of All Percent of All Students StudentsPearson Correlation Passing ELA Passing MathPercent of ELLs Enrolled -.297(**) -.382(**)Sig. (2-tailed) .00000 .00000** Correlation is significant at the 0.05 level (2-tailed).Table 4.2.4Pearson Correlations: 2006 10th Grade TAKS for English Language Artsand Mathematics. Percent of All Percent of All Students StudentsPearson Correlation Passing ELA Passing MathPercent of ELLs Enrolled -.398(**) -.356(**)Sig. (2-tailed) .00000 .00000** Correlation is significant at the 0.05 level (2-tailed). A regression analysis for the data in 2003 is shown in Tables 4.2.5and 4.2.6. The correlation coefficient between the percentage of ELLsenrolled in a school and the percentage of all students passing in 10thgrade TAKS in English Language Arts for 2003 is -0.349. The Pearson rbetween the percentage of ELLs enrolled in a school and the percentageof all students passing in 10th grade TAKS in Mathematics is -0.293.Both correlation coefficients are significant at p < 0.05, two-tailed.
    • 93 The percentage of all students passing in 10th grade EnglishLanguage Arts TAKS can be predicted using the linear regressionequation: Ŷ = 68.71 – 0.70X, where X is the percentage of ELLs enrolledin a school. For example, using the percentage of ELLs data for 2003(X = 8.30), the predicted value for percent of all students passing 10 thgrade English Language Arts is equal to 68.71 – 0.70(8.30) or 62.90.Actual result was 62.87. The percentage of all students passing in 10 thgrade Mathematics TAKS can be predicted using the linear regressionequation: Ŷ = 67.49 – 0.68X, where X is the percentage of ELLs enrolledin a school. A similar example for Mathematics gives a predicted value of67.49 – 0.68(8.30) or 61.85. Actual result was 61.85.Table 4.2.5Coefficients for Percentage of All Students Passing the 2003 10 th GradeEnglish Language Arts TAKS.Model Unstandardized Standardized T Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 68.71 1.89 36.37 .000 Percent ELLs 0 -.70 0.16 -.349 0-4.55 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
    • 94Table 4.2.6Coefficients for Percentage of All Students Passing the 2003 10 th GradeMathematics TAKS.Model Unstandardized Standardized T Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 67.49 2.22 30.43 .000 Percent ELLs 0 -.68 0.18 -.293 -3.74 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS Results of the regression analysis for the data in 2004 are shownin Tables 4.2.7 and 4.2.8. The correlation coefficient between thepercentage of ELLs enrolled in a school and the percentage of allstudents passing in 10th grade TAKS in English Language Arts for 2004is -0.392. The Pearson r between the percentage of ELLs enrolled in aschool and the percentage of all students passing in 10 th grade TAKS inMathematics is -0.351. Both correlation coefficients are significant atp < 0.05, two-tailed. The percentage of all students passing in 10 th grade EnglishLanguage Arts TAKS can be predicted using the linear regressionequation: Ŷ = 73.76 – 0.66X, where X is the percentage of ELLs enrolledin a school. For example, using the percentage of ELLs data for 2004(X = 8.29), the predicted value for percent of all students passing 10 th
    • 95grade English Language Arts is equal to 73.76 – 0.66(8.29) or 68.29.Actual result was 68.28. The percentage of all students passing in 10 thgrade Mathematics TAKS can be predicted using the linear regressionequation: Ŷ = 60.34 – 0.82X, where X is the percentage of ELLs enrolledin a school. A similar example for Mathematics gives a predicted value of60.34 – 0.82(8.29) or 53.54. Actual result was 53.57.Table 4.2.7Coefficients for Percentage of All Students Passing the 2004 10 th GradeEnglish Language Arts TAKS.Model Unstandardized Standardized T Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 73.76 1.74 42.46 .000 Percent ELLs 0 -.66 0.14 -.392 -4.83 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts
    • 96Table 4.2.8Coefficients for Percentage of All Students Passing the 2004 10th GradeMathematics TAKS.Model Unstandardized Standardized T Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 60.34 2.42 24.90 .000 Percent ELLs 0-.82 0.19 -.351 0-4.28 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS A regression analysis for the data in 2005 is shown in Tables 4.2.9and 4.2.10. The correlation coefficient between the percentage of ELLsenrolled in a school and the percentage of all students passing in 10 thgrade TAKS in English Language Arts for 2005 is -0.297. The Pearson rbetween the percentage of ELLs enrolled in a school and the percentageof all students passing in 10th grade TAKS in Mathematics is -0.382.Both correlation coefficients are significant at p < 0.05, two-tailed. The percentage of all students passing in 10th grade EnglishLanguage Arts TAKS can be predicted using the linear regressionequation: Ŷ = 63.61 – 0.55X, where X is the percentage of ELLs enrolledin a school. For example, using the percentage of ELLs data for 2005(X = 7.62), the predicted value for percent of all students passing
    • 9710th grade English Language Arts is equal to 63.61 – 0.55(7.62) or 59.42.Actual result was 59.39. The percentage of all students passing in 10 thgrade Mathematics TAKS can be predicted using the linear regressionequation: Ŷ = 53.97 – 0.83X, where X is the percentage of ELLs enrolledin a school. A similar example for Mathematics gives a predicted value of53.97 – 0.83(7.62) or 47.65. Actual result was 47.68.Table 4.2.9Coefficients for Percentage of All Students Passing the 2005 10 th GradeEnglish Language Arts TAKS.Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 63.61 2.03 31.28 .000 Percent ELLs 0-.55 0.15 -.297 0-3.59 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
    • 98Table 4.2.10Coefficients for Percentage of All Students Passing the 2005 10 th GradeMathematics TAKS.Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 53.97 2.29 23.54 .000 Percent ELLs 0-.82 0.17 -.382 -4.74 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKS To show that the percentage of ELLs enrolled in a school affectedthe percentage of all students passing the 10th grade TAKS tests inEnglish Language Arts and Mathematics, the regression analysis resultsare shown in Table 4.2.11 and Table 4.2.12. The correlation coefficientbetween the percentage of ELLs enrolled in a school and the percentageof all students passing in 10th grade TAKS in ELA for 2006 is -0.398. ThePearson r between the percentage of ELLs enrolled in a school and thepercentage of all students passing in 10th grade TAKS in Mathematics is-0.356. Both correlation coefficients are significant at p < 0.05, two-tailed. The percentage of all students passing in 10th grade EnglishLanguage Arts TAKS can be predicted using the linear regressionequation: Ŷ = 81.85 – 0.55X, where X is the percentage of ELLs enrolled
    • 99in a school. For example, using the percentage of ELLs data for 2006(X = 6.94), the predicted value for percent of all students passing 10 thgrade ELA is equal to 81.85 – 0.55(6.94) or 78.03. Actual result was78.05. The percentage of all students passing in 10 th grade MathematicsTAKS can be predicted using the linear regression equation:Ŷ = 55.70 – 0.80X, where X is the percentage of ELLs enrolled in aschool. A similar example for Mathematics gives a predicted value of55.70 – 0.80(6.94) or 50.15. Actual result was 50.13.Table 4.2.11Coefficients for Percentage of All Students Passing the 2006 10 th GradeEnglish Language Arts TAKS.Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 81.85 1.24 66.28 .000 Percent ELLs 0-.55 0.10 -.398 -5.37 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade English Language Arts TAKS
    • 100Table 4.2.12Coefficients for Percentage of All Students Passing the 2006 10 th GradeMathematics TAKS.Model Unstandardized Standardized t Sig. Coefficients Coefficients B Std. Beta Error1 (Constant) 55.70 2.08 26.78 .000 Percent ELLs 0-.80 0.17 -.356 -4.67 .000a. Predictor: Percentage of ELLs Enrolled in a Schoolb. Outcome variable: Percentage of All Students Passing in 10 th Grade Mathematics TAKSResearch Question – Qualitative After the refinements were done during the pilot study, an open-ended, on-line questionnaire was given to the principals, assistantprincipals, ESL district personnel, ESL certified teachers and non-ESLcertified teachers who were purposively sampled from urban high schoolsin Texas. The major question answered by this study was: What are theanticipated and observed consequences of the statewide testingspecifically, TAKS, on ESL curriculum and instruction as viewed bycertified ESL teachers, non-certified ESL teachers who teach ELLs,school administrators, and district ESL personnel?
    • 101 This major question was explored using the following probes: 1. Why is TAKS given as a statewide test? 2. What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3. What problems have occurred related to or because of TAKS? 4. What changes were caused by this statewide testing? 5. What are your recommendations to improve this statewide testing? 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? Before the above questions for the qualitative study are answered,the following tables give the demographic information regarding therespondents.Demographic Characteristics of the Respondents Demographic information regarding the respondents includedgender, age, current position, highest degree earned, years of experiencein public education and certification(s) achieved. The open-ended, on-line questionnaire for the qualitative aspect ofthe study was intended for 98 respondents. Six principals responded,together with 9 assistant principals, 6 ESL district personnel, 15 ESLcertified teachers and 19 non-ESL certified teachers – a total of 55respondents, for a response rate of 56%. Table 4.3 shows that 69% of the respondents were female and 31%were male.
    • 102Table 4.3Distribution of Respondents by Gender.Gender Frequency Percent (%)Female 38 69Male 17 31N=55 The age of the respondents is shown in Table 4.4. With respect toage, 31% of the respondents were between 41 – 50 years old and 51 – 60years old. The group of 31 – 40 years old comprised 22% of therespondents; 9% of the respondents were older than 60 years old and theyoungest between 21 – 30 years old was 7% of the group.Table 4.4Distribution of Respondents by Age.Age Frequency Percent (%)21-30 04 0731-40 12 2241-50 17 3151-60 17 31> 60 05 09N=55 It is shown in Table 4.5 that the non-ESL certified teacherscomprised 35% of the respondents and 27% were ESL certified teachers.The administrators accounted for the remaining 38%: 16% were
    • 103assistant principals, 11% were principals, and 11% were ESL districtpersonnel.Table 4.5Distribution of Respondents by Professional Position.Current Position Frequency Percent (%)Principal 06 11Assistant Principal 09 16ESL District Personnel 06 11ESL Certified Teacher 15 27Non-ESL Certified Teacher 19 35N=55 Table 4.6 displays that 53% of the respondents have Master‟sdegrees, 40% have Bachelor‟s degrees and 7% have Doctorate degrees.Table 4.6Distribution of Respondents by Highest Degree Earned.Highest Degree Frequency Percent (%)Bachelor‟s Degree 22 40Master‟s Degree 29 53Doctorate Degree 04 07N=55 Considering years of experience in public education, it is shown inTable 4.7 that 40% of those who participated in the study had more than20 years of public school experience. Thirty-one percent had 11 – 20
    • 104years of public school experience, 16% had 5 years or less, and 13% hadbeen in the public schools for 6 – 10 years.Table 4.7Distribution of Respondents by Years of Experience in Education.Years of Experience Frequency Percent (%)1–5 09 166-10 07 1311-20 17 31> 20 22 40N=55 Table 4.8 displays the certifications of the ESL district personnel,principals, assistant principals, ESL certified and non-ESL certifiedteachers who handle ELLs that participated in this study. Some of theseadministrators and teachers had more than one certification. Based ontotal number of respondents, 47% had certification in English LanguageArts, 20% in Mid-Management, 18% in Mathematics, 16% inPrincipalship, and 13% in Special Education.
    • 105Table 4.8Distribution of Respondents by Certification.Certification FrequencyEnglish Language Arts 26Mid-Management 11Mathematics 10Principal 09Special Education 07Bilingual 02Educational Diagnostician 02Biology 02Sociology 02Supervisor 02Reading 02Librarian 02Superintendent 02History 02Technology 01Marketing 1Counseling 1Vocational Home Economics 01Social Studies Composite 01N=55
    • 106 The findings regarding the qualitative portion of the study werepresented as follows: (1) the nine open-ended responses to the on-linequestionnaire; (2) frequency tables showing the emergent themes,frequencies and percentages; and, (3) anecdotal views and opinions ofdistrict ESL personnel, principals, assistant principals, ESL-certifiedteachers and non-certified ESL teachers handling ELLs regarding theissues or concerns. Answers of some respondents belonged to more than oneemergent theme; the total number of answers may have exceeded thetotal number of respondents. The percentage shown after the totalresponses given for each emergent theme was computed based on totalnumber of respondents.Why is Texas Assessment of Knowledge and Skills (TAKS) given as aStatewide Test? Why is TAKS given as a statewide test? Responses are shown inTable 4.9. Of the 55 respondents, 40% viewed TAKS as a tool to gaugeknowledge in the core academic areas. Twenty-nine percent (29%) of therespondents considered TAKS as a means to determine the school‟seffectiveness and performance (Exemplary, Recognized, AcademicallyAcceptable, or Academically Unacceptable), while 16% regarded TAKS asan instrument to appraise the effectiveness of the state curriculum. Only7% indicated that this statewide test is mandated by law and is alignedwith NCLB.
    • 107Table 4.9Why is TAKS Given as a Statewide Test to ELLs?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach (15) NonTeach (19) Tot/%(55)Gauge Skills/TEKS 3 3 3 6 7 22/40School Accountability 0 2 6 3 5 0 16/29Assess State Curri-Std 1 1 0 3 4 00 9/16Mandated by NCLB 1 0 1 0 2 04/7Com Schools/Districts 1 0 1 1 0 0003/5Generate Revenue 1 0 1 0 1 0003/5Test College Prep 0 0 1 1 0 00 2/4Political Appeasement 0 0 0 1 0 00 1/2No Response 0 0 0 0 1 00 1/2
    • 108 One ESL district personnel (EDP-03) gave this opinion about TAKS:“It is administered to measure the knowledge and skills of Texasstudents,” and “it provides a comparative data table for critics.” Anassistant principal (AP-06) supported this view: “TAKS is to help gaugestudents‟ basic skills” and “to make schools accountable for students‟learning.” Two high school principals (P-06 and P-04) referred to the TAKS asan instrument of accountability: “…currently, TAKS is the StateAccountability System test in order to comply with NCLB.” A more comprehensive response regarding the purpose of TAKSwas given by non-ESL certified teacher (non-ESL-14) who handles ELLs: “At the state level, the TAKS is the state education agency‟s tool for determining the accreditation status of schools and school districts…With the enactment of the No Child Left Behind Act, the TAKS is now being used also as a tool, among other indicators, for determining the attainment or non-attainment of an adequate yearly progress (AYP). It is expected that a school carrying standards-based curriculum and intervention, especially one that is predominantly attended by minority populations, would show improving performance until the targeted year 2013 - 2014 when all children have met standards, which is the essence of the phrase „No Child Left Behind.‟ The TAKS performance is disaggregated into subgroups, so that again, there is a way of determining whether
    • 109 achievement gaps are being bridged. Much of the accountability for student improvement on knowledge and skills in the four academic areas falls on schools. The other indicators of adequate yearly progress are completion rate and attendance.” In addition to being an instrument of accountability, respondentssuggested that assessment results provide data for comparative analysis.According to one assistant principal (AP-09), TAKS is given “tocompare/contrast statewide similarities and differences” in achievementlevels “between schools” and across districts. During one of the focus group sessions, teachers gave thesecomments regarding TAKS: “The test administration is the same for all students. We have to follow the guide from the state. But, getting the kids ready for the test is different. We don‟t do anything differently, but we should. It‟s going to take more than what we are doing right now to ensure these kids (ELLs) graduate with the same diploma, meeting the same standards, as the general population.” Knowing what TAKS is for, does not necessarily translate intounanimity in terms of agreeing on what schools must do to respond tothe challenge that the test brings. Respondents‟ opinions however,indicate the sense that schools need to adapt, and do things differently ifthey expect to achieve some level of parity between the TAKSachievement scores of ELLs and their non-ELL counterparts.
    • 110Anticipated Results of ELLs on English Language Arts and MathematicsTAKS Table 4.10 shows the responses to the following question: Whatare the anticipated results on the statewide testing, specifically EnglishLanguage Arts and Mathematics portion of the Texas Assessment ofKnowledge and Skills (TAKS), on English language learners (ELLs)?Almost half (49%) of those who responded expected ELLs to have at leastaverage scores in TAKS. However, 16% observed ELLs‟ performance to belower than non-ELLs, and 15% projected low results for ELLs because oftheir lower English language skills.
    • 111Table 4.10What are the Anticipated Results of Statewide Testing for ELLs?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Average Scores 1 2 3 3 7 16/29Everyone Pass/Meet AYP 2 1 3 3 2 0 11/20Lower than Non-ELLs 1 2 0 3 3 009/16Low: Language Barrier 1 1 1 3 2 08/15Curriculum Improvement 2 0 0 0 2 0004/7Dismal 0 0 1 0 1 0002/4Cause Dropouts 1 0 0 1 0 00 2/4Gap in Scores to Lessen 1 0 1 0 0 00 2/4Team Effort Improvement 0 0 0 0 2 00 2/4Political 0 0 0 1 0 00 1/2Modified Diploma 0 0 0 1 0 00 1/2Not Sure 0 0 0 1 0 00 1/2
    • 112 One high school principal (P-04) anticipated that the: “results ofstatewide testing are that all students will be expected to meet minimumstate standards.” Another high school principal (P-02) expressed withsome sense of exasperation that: “English language learners tend to score extremely low on the ELA and Mathematics portions of the TAKS. The state and the federal governments expect us to work miracles with these groups of students. If a student is in the U.S. for a year or two, he/she needs extra remediation to get a significant gain in TAKS scores. He may not necessarily pass the TAKS, but he will make gains. Keep in mind the NCLB ruling. Making any gains is not enough. The students must perform at 70% or better on all four sections of TAKS. At this point, all administrators in poverty-stricken areas are in fear of losing their jobs due to the low TAKS scores. We are indeed under extreme pressure!” One principal (P-05) indicated that the TAKS “will have a negativeimpact on the English language learners because of their difficulty inreading and writing in English.” The same negative outlook wasexpressed by an assistant principal (AP-03) who bluntly predicted that:“ELL students will fare worse than the general population.” A non-ESL teacher handling ELLs (Non-ESL-14) proposed ameasure to address the feared, and somewhat expected under-achievement of ELL Learners:
    • 113 “Title I programs are instituted to address the intervention needs of ELLs so that they perform well in statewide testing. The said federal assistance is aimed at incrementally improving the performance of disadvantaged groups, which include the ELLs and the ethnic population they comprise. In my particular school, intervention translates into identifying those who are at risk in this group, breaking down their previous performances into specific areas and conducting the needed mentoring.” During a focus group interview, one member of the groupexpressed this view regarding the anticipated results TAKS : “I think that if we are going to require ELLs to be as successful on TAKS as any native English speaker then those students need specific language classes in school. ELLs need English classes whose sole purpose is teaching them to understand and speak English, with basic grammar skills included. In addition to a language class, they do still need a literature-based class in which they are learning to read and analyze text, as well as write both creatively and analytically. Also, having tutoring available, either built into the school day, or after school, during which ELLs practice speaking and listening in English would do a world of good. Teachers would not need to be the only tutors either, we could tap into a college‟s community service group, or even take advantage of peer tutoring.”
    • 114Actual Results of ELLs in English Language Arts and Mathematics TAKS Table 4.11 summarizes the responses for the question: What arethe actual results of TAKS, specifically English Language Arts andMathematics, on ELLs? Responses were related to how the ELLs in theschools actually performed on TAKS. Results were more negative thanpositive.Twenty-two percent (22%) observed dismal or failing performance, 18%noticed performance of the ELLs to be lower than non-ELLs and 11%mentioned that low results frustrate ELLs.
    • 115Table 4.11What are the Actual Results of Statewide Testing for ELLs?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Dismal/Failing 0 1 5 1 5 12/22Lower than Non-ELLs 3 1 1 3 2 0 10/18Average Results 0 2 2 2 3 009/16No Response 0 0 0 4 4 08/15Frustrated ELLs 3 0 0 2 1 0006/11Better than Average 0 2 1 0 0 000 3/5Don‟t Know 0 0 0 0 2 00 02/4Gap Narrowed 0 0 0 0 2 00 02/4Showed Areas to Improve 0 0 0 1 0 00 01/2Political 0 0 0 1 0 00 01/2Teacher Teach TAKS 0 0 0 1 0 00 01/2
    • 116 Actual results in TAKS by ELLs are shared by an assistantprincipal (AP-03): “ELL students do fare worse than the generalpopulation.” The same observation was given by a non-ESL certifiedteacher (Non- ESL- 03): “Actual results show that ELLs perform poorer than native speakers and naturally so for they are very much handicapped as far as their understanding of the intricacies of the language is concerned. Their scores result in schools being „below standard‟, so to speak.” A non-ESL certified teacher (Non-ESL-14) offered a contrastingview by sharing the actual experience in their district: “I do not have the exact statistical report as far as the performances of ELLs are concerned, but my school district came up with a report lauding the improvement across grade levels in Mathematics and English Language Arts among ELLs, except in 9th Grade Mathematics. The district superintendent also mentioned about the narrowing of the achievement gap, and cited the percentage points of improvement.”Intended Consequences of TAKS for ELLs Table 4.12 summarizes the responses to the question: What arethe intended consequences of this statewide testing as it relates to ELLs?With respect to the intended consequences of TAKS, 18% of therespondents expressed the understanding that statewide testing is
    • 117intended to eventually result in ELLs performing as well as the rest of thestudents and 18% agreed that TAKS is a tool to assess ELLs. Elevenpercent (11%) of the respondents expressed confidence that ELLs canimprove academically and eventually join the mainstream, and 11%demonstrated awareness that TAKS may improve the graduation rate.Teachers (9% of the respondents) expressed their wish for an opportunityto teach ELLs individually with the end in mind of improving theirperformance in TAKS.
    • 118Table 4.12What are the Intended Consequences of TAKS for ELLs?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Same Level of Expectations 1 2 0 4 3 10/18Assess ELLs 1 1 1 4 3 0 10/18ELLs Improve-Mainstream 2 0 1 1 2 0006/11Improve Graduation Rate 0 1 1 2 2 006/11ELLs Learn English ASAP 0 0 2 2 2 00 06/11School Accountability 1 1 1 1 1 0000 5/9Individual Teaching of ELLs 0 0 0 2 3 00 005/9Equal Learn-Opportunities 0 1 1 0 1 0 003/5Improve Teacher Practices 0 0 1 1 1 00 003/5Not Sure 1 0 1 0 1 0 03/5No Response 0 0 0 0 1 00 1/2
    • 119 An assistant principal (AP- 09) stated that statewide testing isintended to ensure equality of learning opportunities for minority groups.Another assistant principal (AP-03) saw the tests as eventually resultingin the elevation of standards and the improvement of ELL students‟English language skills. A number of non-ESL certified teachers heldsimilar views: “With this statewide testing, ELLs can show how much they have gotten into the „mainstream‟ of the American educational system. In so doing, the government will get an „assurance‟ that graduates of this educational system (or future adults of this society) are competent and can function well in… this system” (Non-ESL-03). “The ultimate objective of this educational reform is for everyone... particularly minority populations, to be educationally competent so that they become more and more socio-economically integrated into the American mainstream” (Non-ESL-14). While the respondents demonstrated competent awareness of theintended consequences of the TAKS, a few were quick to point out thechallenges that testing poses. A principal (P-02) stated: “I agree with the No Child Left Behind. We care about each and every student, but you can‟t compare a 5th generation student with a student who has only experienced a month or two in the U.S.A. and cannot speak the English language or may not have had formal education in his/her home country. These students are an
    • 120 extremely difficult challenge. This challenge is rarely seen in the suburban schools.” On the other hand, one non-ESL certified teacher (Non-ESL-09)observed that the intended consequence of the TAKS: “Is to let ELL students stay in high school for more than 4 years because they can not graduate without passing the TAKS exit level…it should serve as a challenge to ELL kids to strive more and work harder because …we all agree that at some point, Mathematics in itself is a language…so anybody who has this class is learning a new language… ELLs should not use language as a barrier to learning Mathematics.” An English teacher of a focus group expressed this opinionregarding how statewide assessment affects ELLs: “Statewide assessments that are intended for non-ESL students have detrimental effects on the ELL populations. Many are indeed good students, but they get their results back and only see failure. The educator also sympathizes because he/she sees firsthand how far the student has come, but there is no state test for progress. It becomes frustrating for all parties involved. Sometimes administrators assume the teacher is not doing his/her job because the scores come back so low.”
    • 121What Has Happened to TAKS, As It Relates to ELLs What has happened because of TAKS, as it relates to ELLs?Responses given to this question are shown in Table 4.13. Twenty-sixpercent (26%) of the respondents reported that there is a high failure rateamong ELLs. Other respondents observed that ELLs experienced adiminished self-esteem because of low scores (reported by 11% of therespondents) and tests caused frustration and exasperation on the partof ELLs (also seen by 11% of the group). There is also pressure on ELLsand the schools they attend to improve performance as noted by 11% ofthe respondents. Eleven percent (11%) of the respondents reported thatthere is now an overwhelming emphasis on test performance in theircampuses.
    • 122Table 4.13What Has Happened to ELLs Because of TAKS?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Higher Failure Rate-ELLs 1 1 5 3 4 14/26Dropout Rate Increased 2 1 1 1 2 0 7/13ELLs Have Low Self-Esteem 0 1 1 1 3 006/11Pressure on School/ELLs 0 1 1 1 3 06/11Emphasis Test Performance 1 0 0 3 2 00 6/11Frustration/Exasperation 1 1 0 2 2 00 6/11Focus on Needs of ELLs 1 1 0 1 1 00 4/7Teachers Teach to TAKS 0 0 0 2 2 00 4/7Continue Testing Changes 0 0 1 0 2 00 03/5ELLs Repeat TAKS Often 0 0 0 2 0 00 02/4Acceptable Test Scores 0 1 0 1 0 00222/4Get Rid of TAKS 0 0 1 1 0 00 2/4No Response 0 0 0 1 1 00 2/4
    • 123 District personnel (EDP-03) noted that “many of the ELL studentshave dropped out, fallen behind and in some instances stopped caring.Those who successfully passed the TAKS are concerned with their familyand friends, so they share the failure. An assistant principal (AP-02)stated that TAKS has resulted in an “increase in dropout rate, low selfesteem.” An ESL teacher (ESL-01) has observed that “many ELLs fear TAKSand do not think they will ever be able to pass it… many drop out ofschool altogether.” Another ESL teacher (ESL-05) noticed somethingsimilar: “Every year ELLs either pass and exit the program, or fail… theyexperience a sense of failure.” “The test has caused frustration andexasperation on the part of many students because the stakes are toohigh. They are only judged on this one test” (ESL – 10). Similarly, a non-ESL teacher (Non- ESL- 03) observed that: “Because of TAKS (and its results), ELLs have become the “trail enders” (the ones at the end of the trail). Not performing well in these tests and being “left behind” by the “better” students has given them lower self-esteem and feelings of frustration. Failing the test one too many times may result in feelings of frustration and, possibly, rejection on their part.” An ESL teacher (ESL-07) observed that “ELLs have beenpenalized, a large portion must re-take TAKS multiple times in order to
    • 124pass, and schools have been held accountable for the failure of thatpopulation of students.” A more optimistic response was expressed by a non-ESL teacher(Non- ESL-14) regarding what happened to ELLs because of TAKS: “School administrators and educators have become more aware of the standards and expectations that students must reach for them to become competent human resource in the workplace and society. Not only has there been a gap among ethnic populations, with the ELLs at the disadvantaged end of the spectrum; there has also been a gap between competencies of exiting high school students and the actual skills and scholastic aptitude required in postsecondary education. Falling on the shoulder of the academic workforce, the TAKS has pushed these school stakeholders to intensify their efforts of addressing the needs of ELLs. Student expectations and academic objectives are being laid out more clearly, and their assessments more meticulously scrutinized with students, as part of instruction, in order for them to become more aware and self-reflective in their learning. There is a culture of concern for statistical performance, broken down furthermore into numbers vis-à-vis chunks of knowledge and skills in a specific subject area. Even ELLs have become familiar with the language of testing.”
    • 125Problems of ELLs, Due to TAKS What problems have occurred, if any, to ELLs related to or becauseof TAKS? Responses are summarized in Table 4.14. Of the 55respondents, 24 % mentioned the higher dropout rate as one of theproblems experienced by ELLs due to TAKS. At least one from each groupexpressed this problem. Graduation rate decreased for ELLs due to thisstatewide test - this was echoed by 22% of the respondents. Anothernegative aspect is perceived to be the low esteem of ELLs as seen by 13%of the group; 15% of the respondents realized the need to support ELLs.
    • 126Table 4.14What Problems Have Occurred for ELLs Due to TAKS?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Higher dropout rate 3 2 1 4 3 13/24Low graduation rates 2 0 1 4 4 0 11/20Support for ELLs 0 1 1 4 2 008/15Low self-esteem 0 1 1 3 2 07/13Instruction is test oriented 1 0 0 2 1 00004/7Increased achievement gap 0 0 2 1 1 000 4/7Ostracized due to low scores 0 0 1 0 2 00 03/5Tutorials/staff development 0 1 0 1 1 00 03/5True ability not measured 0 0 2 0 1 00 03/5No problem 0 2 0 0 1 00 03/5Limited reading ability 0 0 0 0 2 00 02/4Rush ELLs to learn English 0 0 0 0 1 01001/2No Answer 0 0 0 0 1 01001/2
    • 127 Problems experienced by principals included the following: “Somehave dropped out; others have done well enough to finish high school butfear college. Others have lost all hope and courage to continue” (P-03).“The results of TAKS have caused the English language learners to feelas if they are failures” (P-05). A non-ESL teacher (Non-ESL-03) observed the same problem:“ELLs … may just disengage themselves from any school activities andstudies, and may, ultimately, check out of school.” This was echoed byan assistant principal (AP-07) who observed that ELLs “leave high school,complete all credits but do not receive diploma due to failing statewideexam.” Though challenged, a principal (P-02) remained optimistic: “Some of our English language learners‟ hopes and dreams of passing the state exams never become a realization, but we don‟t give up on them. We keep them and continue to work with them individually so that they may succeed.” The ELLs‟ performance in TAKS was explained by a non-ESLteacher (Non-ESL-14): “ELLs enroll into their grade levels not quite equipped with the prerequisite skills to perform” according to standards. “Even with the administration of the TAKS in the native language of the ELLs, there still exists an academic gap, because the testing language is not essentially native but academic, an orientation with which ELLs are not familiar. Only 4 years old, the TAKS as a federal tool
    • 128 for adequate yearly progress is battling against a culture (focused on) economic survival. The TAKS measures scholastic aptitude and academic achievement - attributes that make up the profile for college readiness, or postsecondary education.” It was also observed that ELLs are more likely to need employmentto survive. ELLs predominantly come from the working class, and theyview attendance in school as a bureaucratic legal requirement ratherthan as an opportunity to develop skills necessary for social mobility.Changes Which Occurred as a Result of Statewide Testing, as TheyRelate to ELLs Table 4.15 displays the responses to the question: What changeshave occurred as a result of statewide testing as it relates to ELLs? Dueto the low performance of ELLs, 35% of those who responded to thequestionnaire saw the need to focus and intervene on behalf of ELLs.Although 16% observed no changes because of TAKS, 15% realized thatthere will be more pressure at home and school to help ELLs pass TAKS.Thirteen percent (13%) of the respondents affirmed their commitment tobe accountable for the ELLs.
    • 129Table 4.15What Changes Have Occurred for ELLs Due to TAKS?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Focus/Intervention for ELLs 2 3 2 4 8 19/35None 0 0 3 3 3 0 9/16More pressure: school/home 1 1 1 3 2 008/15Accountability for ELLs 1 2 2 1 1 07/13Teacher development 0 1 1 2 1 000 5/9Uncertain 2 0 0 2 1 000 5/9Emphasis: test performance 2 0 0 0 1 00 3/5Positive pressure to perform 0 0 0 1 1 00 2/4Instruction is test oriented 0 0 0 0 2 00 2/4Fewer Exemptions 1 0 0 1 0 00 2/4Need effective leadership 0 0 1 0 2 00 1/2Not welcome: public schools 0 0 0 0 1 010 1/2
    • 130 In response to what should be done regarding the expectedchanges, a non-ESL teacher (Non-ESL-11) suggested that “the changesthat have occurred include more intervention programs in classrooms,on campuses and in districts.” This move was also supported by anothernon-ESL teacher (Non-ESL-14): “There is a growing number of ELLs being motivated to perform according to standards. These are the ones who can actually see beyond high school graduation upon passing the exit-level TAKS. Opportunities for postsecondary education exist along with standardized testing, and so the high school journey for these ELLs seeing the long-term meaning of the TAKS gives them a greater sense of direction. The TAKS has also created positive pressure among this group of ELLs in that they have become aware of specific academic competencies they are expected to attain, thereby creating in them greater self-efficacy and confidence for postsecondary education.” An assistant principal (AP-06) observed that: “The changes that have occurred in some schools have been teachers receiving more staff development in how to instruct students whose first language is not English. More responsibility has been placed on teachers in all classes to help strengthen the students‟ ability to pass the test.”
    • 131 A non-ESL teacher (Non- ESL- 09) also saw that “changes inteaching strategies are affected…teachers should be able to teach usingEnglish as a medium of instruction and yet accommodate the ELLs intheir classes.” A principal (P-02) offered this strategy to cope with changes due toTAKS: “We are focused on each child. The teachers are aware of theirstrengths and weaknesses. Teachers meet with administrators to talkabout the students and what strategies they will use to help the ELLattain the level of academic achievement that the state and the schoolrequire.”Recommendations to Improve Performance of ELLs in Statewide Testing Respondents gave their recommendations on how to improve theoverall performance of ELLs in TAKS; these are shown in Table 4.16.Recommendations involved: (1) different/fairer test; and, (2) betterassistance from the school through the teachers and the curriculum.Twenty percent (20%) of the respondents clamored for a different and fairtest, while 9% opted for a test to be given 5 to 7 years after ELLs startedschooling in the country. Eighteen percent (18%) recognized the need forbetter prepared teachers; 15% requested for modifications in teaching;and 11% proposed a paced curriculum for ELLs.
    • 132Table 4.16What Recommendations are suggested for Improvement of ELLs Performance on TAKS?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Different/fairer test 1 0 0 5 5 11/20Better prepared teachers 1 3 0 4 2 0 10/18Teaching modifications 2 2 1 1 2 008/15Paced curriculum 2 0 1 0 3 06/11Test ELLs later (5-7years) 0 0 1 2 2 000 5/9Increase overall performance 0 0 1 2 2 000 5/9Intense English programs 0 1 3 0 0 444 4/7Bilingual teachers for ELLs 0 0 0 0 3 00 3/5Monitor ELLs 0 1 1 1 0 00 3/5Test preparation classes 1 0 0 0 1 00 2/4Smaller classes-create PLCs 0 0 1 1 0 00 2/4Involvement of parents 0 0 0 0 2 010 2/4
    • 133 ESL district personnel (EDP-03) recommended: “State legislators(individuals who formulate the TAKS test) need to speak with ELLteachers as well as the ELL students” to develop a more appropriate testwhich measures achievement. Two ESL teachers (ESL-06 and ESL-07) offered the followingchanges in the administration of the TAKS to ELLs: “There should be varying levels of the TAKS test for ELLs. When students fail the TAKS, it‟s not because they are incapable of passing…but the test is not a very ELL-friendly exam. If TAKS is „supposed‟ to be a basic skills test for non-ELLs, then a test should be devised to test the basic skills that ELLs have acquired since they have learned English. It is not fair that a student that has been speaking English their entire life takes the same test as a person that has only been speaking English 3 or more years in school.” “Have an „alternate‟ state test for the students who enter the country/school at such a late age, which they realistically cannot be expected to have enough of a grasp of English to pass the TAKS. This of course would be a small portion of the ELLs. ” A non-ESL teacher (Non-ESL-16) proposed “that ELLs be given atleast 5 years of residency before they are given a statewide test. Teachersneed to be certified to teach ELLs so that the “problem” can be addressedproperly.”
    • 134 An assistant principal (AP-02) recommended this plan of action:“allow them more time to learn the English language beforeadministering the TAKS Exit Level Test; schedule them in intenseEnglish/Reading classes for 75% of the school day.” An even more drastic move was suggested by a non-ESL teacher(Non-ESL-17) who suggested that we “do away with TAKS as a state test”altogether. Another non-ESL teacher (Non- ESL-14) recommended thefollowing regarding TAKS: “Curriculum and instruction must translate standards into classroom experience that takes into account their socio-cultural milieu, their prior knowledge and skills and their linguistic repertoire. It would help if those who are academically lagging behind, or enrolled at a level several years above their actual level, are given remedial courses so that they do not necessarily have to study material that is incoherent to them, but would bring them up from where they are to where they are supposed to be. Smaller classes or intervention groups would perhaps ensure that greater attention is given to the individual ELL‟s need. Since intrinsic motivation for academic study is undermined by the survival mode, perhaps a culture of career orientation can be instituted where professionals from the community could conduct career activities in schools. This will indirectly motivate towards ELLs
    • 135 seeing the significance of standardized testing. Increased involvement of parents is a big factor in this intrinsic motivation.” Recommendations by a focus group included the following: (1)“Much, much smaller classes for one thing, but it never happens becauseit always comes down to money”; (2) “Home and community push forexcellence. Adult business tutors brought in on a weekly basis”; and (3)“I believe that technology, certain computer programs, may be beneficialto the ELL population.”Recommendations of Greatest Value for ELLs‟ Success on StatewideTesting From the recommendations given, the respondents identified whatwill be of greatest value for ELLs‟ success on statewide testing. Thesesuggestions are given in Table 4.17. Intervention for ELLs wasrecommended by 27% of the respondents, while 16% emphasized qualityinstruction, and 13% suggested a modified test. A more intensive Englishprogram was proposed by 9% of the group, while 9% also observed theneed for help from home or for ELLs to have meaningful tutoring inschool.
    • 136Table 4.17What are the Recommendations, with the Greatest Value, are offered for ELLs Success on TAKS?Emergent Themes Dist.(6) Prin (6) A.Prin (9) ESLTeach(15) NonTeach(19) Tot/%(55)Interventions 2 3 1 3 6 15/27Quality Instruction 1 2 1 0 5 0 9/16Modified Test 0 0 1 3 3 007/13Intensive Programs for ELLs 1 0 2 2 0 0 5/9Tutoring 0 0 0 2 3 0005/9More ESL Teachers 0 0 0 2 2 0004/7Practice Test 2 0 1 0 1 00 4/75 or More Years Residency 0 0 0 1 3 00 4/7Bilingual/ESL Classes 0 1 2 0 0 00 3/5None/Don‟t Know 0 1 1 0 1 00 3/5Smaller Class 0 0 0 1 0 00 1/2Modified Diploma 0 0 0 1 0 00 1/2Effective School Leadership 0 0 1 0 0 00 1/2
    • 137 A principal (P-03) considered this scenario as beneficial to theELLs: “Better trained ESL teachers, more training for the traditional teachers and training for administration on an assets-based model rather than deficit. Also setting up more parent and community involvement. Parents do care and want the best for their children but they need to feel comfortable coming to school and made to feel welcomed.” An assistant principal (AP-09) suggested that: “Bilingual/ESLclasses should be continuously offered. Certified Bilingual/ESL teachersshould be recruited to offer even more assistance to these students.” Thissuggestion finds support from an ESL teacher (ESL-03) who stated that:“ELLs must be taught the skills that will allow them to assimilate into anew society at a rigorous level. The test should reflect their needs andalso be more culturally sensitive.” A non-ESL teacher (Non-ESL-14) averred that: “The remedialintervention program for ELLs whose academic skills fall below theiractual grade level would greatly help in an immediate manner.” Yetanother non-ESL teacher (Non-ESL-17) suggested that “ESL teachersand regular teachers focus on teaching skills that would be mostbeneficial to the students based on their future aspiration (work orcollege after high school).”
    • 138 Regarding statewide testing, an assistant principal (AP-06)proposed that: “The test should be written in the language the student speaks. Are we testing for academic skills? Or, are we trying to hold some schools more accountable than others when giving tests? Schools that have a higher minority rate must have teachers who are more specialized in special education or bilingual education.” During one of the interviews, a principal stated a rather grimscenario involving ELLs: “Standards are going to be difficult for them to achieve due to their lack of language and academic skills. It takes an ELL student five to seven years to acquire language skills adequate for the rigor of high school curriculum. These students eventually pass but not before causing tremendous challenges on our respective campuses. These students show up on our „dropout lists.‟ They have a huge impact on all our student data. For the most part these students come from dysfunctional homes. The father is in Mexico. Mother is here trying to raise a house full of kids- often forced to move from place to place due to financial difficulties. These students seldom stay in one school very long. I think the family unit should move here together and stay until the student graduates from high school.”
    • 139 Another principal offered a radical suggestion regarding what canbe done to handle ELLs regarding the issue of statewide testing: “If I could, I would send them (ELLs) all away from this school. They would all be placed in a school designed to introduce, develop, and build their language and culture skills, they bring our scores down. That shouldn‟t be any surprise. Their skills are weak. Their language is weak. Many of these students develop discipline and attendance problems. Our ISD provides a „new comers‟ program for ESL students. The campus is a separate campus-away from the main high school. Students remain on the „new comers‟ campus for about six months. Then, when they are ready, the students are transitioned onto the main campuses. They ruin our state scores; and wreak havoc on AYP scores. We have a large population of these kids. Our school data indicates that we have 19 percent ESL students. That does not, however, include the monitored kids. Most ESL students eventually pass the state exam but we will never get above „acceptable‟ status if we continue to try to teach and educate these kids on our campuses, within the regular population. The faculty and staff must work ten times harder than schools with few ESL kids.”
    • 140 A principal explained how a strategy worked in their campus tohelp ELLs: “ELL students who continue to be monitored are placed in the appropriate English Language Arts course and are served by certified ESL instructors. As with all of our students, tutorial assistance is provided one hour each week after school for the four key academic content areas. Tutorials begin the second week of school and continue throughout the year (generally increasing in number prior to TAKS testing in the spring). Teachers closely monitor student performance and parents are contacted at the first sign of difficulty. Informal parent teacher conferences are held, and the student is encouraged to take every advantage of the assistance provided by the instructional staff. Students are held accountable for their study time. Note that our students are expected to study approximately one and a half hours per school night. Students and parents sign a contract to this effect prior to being accepted at the school. If informal parent conferences are not successful in providing the assistance and motivation to improve academic success, the Intervention Assistance Team is employed.” ELLs enrolling in a school during anytime during the year is areality school administrators and teachers have to face in public schools.
    • 141They have to be ready to map out a course of action for each ELL toprovide guidance and education during the stay of that student in theirschool. DiscussionResearch Question - Quantitative Is there a relationship between the percentage of English languagelearners enrolled in a school and the percentage of all students passingthe 10th grade TAKS tests in the core areas of English Language Artsand Mathematics given in 2003, 2004, 2005, and 2006? The descriptive statistics showing the means of the 10th gradeTAKS for English Language Arts and Mathematics do not indicateimprovement in performance despite the decrease in the percent of ELLsenrolled in a school. Percent ELLs was 8.30% in 2003, 8.29% in 2004,7.62% in 2005 and 6.94% in 2006. However, it is in the computations forPearson r correlation that the significant relationship was determined. Using the SPSS software package for the TEA data of the sampledschools, the obtained Pearson r correlation coefficients were: -0.349 in2003, -0.392 in 2004, -0.297 in 2005 and -0.398 in 2006 for EnglishLanguage Arts; -0.293 in 2003, -0.351 in 2004, -0.382 in 2005, and-0.356 in 2006 for Mathematics. All of the computed Pearson rcorrelation coefficients were significant at the 0.05 level, two-tailed. All ofthe obtained Pearson r correlation coefficients were negative for bothcontent areas; this inverse relationship indicated that as the percentage
    • 142of ELLs increased, the percentage of all 10th grade students passing ineach of the core areas of English Language Arts and Mathematicsdecreased. A regression analysis using the percentage of ELLs enrolled ina school as a predictor variable resulted to a linear regression equationwhich can predict the percentage of all students passing in 10 th gradeTAKS in English Language Arts and Mathematics. Any school which has ELLs should be aware of the number ofthese types of students. If there is a surge in the influx of ELLs sometimeduring the year, the school should carefully plan on how to prepare themfor future testing. Both curriculum and instruction for ELLs may be putunder review for improvement to better address this growing populationof ELLs.Research Question - Qualitative What are the anticipated and observed consequences of thestatewide testing specifically, TAKS, on ESL curriculum and instructionas viewed by certified ESL teachers, non-certified ESL teachers whoteach ELLs, school administrators, and district ESL personnel? This major question was explored using the following probes:1. Why is TAKS given as a statewide test? Respondents gave the following answers: (1) TAKS is given as a toolto gauge knowledge in the core areas or what is supposed to be taught;(2) TAKS is considered as a means to determine the school‟s status(Exemplary, Recognized, Academically Acceptable, or Academically
    • 143Unacceptable); (3) TAKS is regarded as an avenue to assess the statecurriculum or standards; and (4) this statewide test is mandated by lawand is aligned with NCLB. The reasons why TAKS is given have beenunderstood by the different school personnel.2. What are the intended consequences of this statewide testing? Respondents expressed the understanding that statewide testing isintended to eventually result in ELLs performing as well as the rest of thestudents. Respondents are confident that ELLs can improve academicallyand eventually join the mainstream; they are also aware that TAKS is arequirement for graduation. Teachers expressed their wish for anopportunity to teach ELLs individually with the end in mind of improvingtheir performance in TAKS. School personnel affirmed their commitmenton improving the situation of ELLs in school.3. What problems have occurred related to or because of TAKS? Respondents identified the following as problems encountered byELLs due to TAKS: (1) higher dropout rate; (2) graduation rate decreasedfor ELLs; and (3) low self-esteem of ELLs. These problems are rooted onthe difficulty ELLs face in passing the TAKS tests.4. What changes were caused by this statewide testing? Due to TAKS, schools experienced the negative reality that there isa high failure rate among ELLs. Other unpleasant realities included lowself-esteem because of low scores and tests caused frustration andexasperation on the part of ELLs. There is mounting pressure on the
    • 144school and ELLs and emphasis is placed on test performance. Changesrequire action to improve the situation of the school, particularly thoseaspects which are related to the ELLs. Low scores have to be improvedand ELLs need counseling on ways on how to have a positive view ofanything one meets in life.5. What are your recommendations to improve this statewide testing? Recommendations involved two main factors: (1) deferment of thetest, possibly a different but fair test; and, (2) better assistance from theschool through the teachers and the curriculum, modifications inteaching and possibly a paced curriculum for ELLs. Since onerequirement of NCLB is testing and results indicate progress, there isneed for school personnel to continually monitor ELLs and devisestrategies to better help these students.6. What needs to be done for the ESL students to improve theirperformance in general and specifically for this statewide test? The following recommendations were seen as more helpful to ELLs:Specific interventions for ELLs, quality instruction, a more intensiveEnglish program, help from home and meaningful tutoring in school.Both the school and the home, together with the community have to bepartners in getting involved with ELLs.
    • 145 Summary Since the researcher utilized the explanatory design of mixedmethods, both quantitative and qualitative aspects were considered inthis study. From the data gathered from TEA and after utilizing the SPSSsoftware package for Pearson r correlation, the obtained results indicatethat there existed significant, negative relationships between the percentof ELLs enrolled in a school and the percentage of 10th grade studentspassing in the English Language Arts and Mathematics TAKS tests givenin 2003, 2004, 2005, and 2006. The null hypotheses were then rejected.The regression analysis provided the linear regression equations topredict the percentage of all students passing in 10 th grade TAKS tests inELA and Mathematics using the percentage of ELLs enrolled in a schoolas the predictor variable. For the qualitative dimension of the study, the respondents weremostly certified in English Language Arts, Mathematics, and SpecialEducation for teachers and some process of administrative certificationfor ESL district personnel, principals and assistant principals. As a statewide test, TAKS was seen as a tool to gauge knowledgeand skills in the core areas. According to the respondents, students whotook TAKS were expected to pass and have at least average scores.Unfortunately, most of the respondents saw dismal or low results in theirrespective schools, especially among ELLs. Consequently, focus had been
    • 146centered on this group of students and recommendations includedtesting them at a later date by using a modified test, and havinginterventions to improve the overall performance of ELLs. Administrators and teachers realized the need to improve thequality of instruction and provided interventions especially gearedtowards improving the academic performance of ELLs. It is hoped thatboth instruction and curriculum for ELLs were directed towardsimproving the plight of ELLs.
    • CHAPTER VSUMMARY, CONCLUSIONS, IMPLICATIONS AND RECOMMENDATIONS Summary The purpose of this study was to determine the impact of high-stakes testing on English Language Learners (ELLs). This was shown inboth the quantitative and qualitative dimensions of the study. Bothquantitative and qualitative dimensions of the study provided the statusof high-stakes testing as it affected ELLs and how it influenced efforts inschools to improve performance of students, particularly ELLs. Dataobtained from Texas Education Agency (TEA) were used to determinewhether there was a relationship between the percentage of ELLsenrolled in a school and the percentage of all students passing the 10thgrade Texas Assessment of Knowledge and Skills (TAKS) tests in the coreareas of English Language Arts and Mathematics given in 2003, 2004,2005, and 2006. To support the qualitative aspect, this study exploredwhat certified English as a Second Language( ESL) teachers, non-certified ESL teachers who teach ELLs, administrators, and district ESLpersonnel viewed as the impact that high stakes standardizedassessments had on ELLs, ESL curriculum and instruction. Review of literature included the important consideration of the NoChild Left Behind Act (NCLB). NCLB holds states using federal fundsaccountable for student academic achievement. States are required todevelop a set of high-quality, yearly student assessments that include, at 147
    • 148a minimum, assessments in Reading/Language Arts, Mathematics andScience. NCLB requires states to report Adequate Yearly Progress (AYP)for all students and for subgroups, including students with limitedEnglish proficiency (Abedi, 2004). Phrases such as “studentachievement,” “proficiency,” “raised expectations” and “testing” areimplications of NCLB. Certainly, the focus on holding schoolsaccountable for student achievement on standardized tests sets NCLBapart from previous versions of the law. (Guilfoyle, 2006). The focus of the study is the ELLs. The term “English languagelearner” is a recent designation for students whose first language is notEnglish. This group includes students who are just beginning to learnEnglish as well as those who have already developed considerableproficiency. The driving force behind including English language learnersin statewide accountability testing is the legislation requiring it. In orderto continue to receive Title I funds through NCLB, states must set highstandards for all students and implement accountability systems tomeasure progress towards those standards. NCLB specifically states thatEnglish language learners must be included in statewide accountabilitytesting, that their scores must be disaggregated so that it can be seenhow they are achieving as a subgroup, and that the assessment systemmust accommodate their linguistic needs (“NCLB”, 2002).
    • 149 High-stakes testing -- using standardized scores to imposeconsequences affecting teachers and students – has been embracedwidely in recent years as a way to hold educators and studentsaccountable for their performance. Experts say the movement is one ofthe most significant shifts in U.S. education in decades (Whoriskey,2006). The goal of statewide accountability testing for English languagelearners (or for all students for that matter) is to improve standards-based practices. The intended “washback” of including English languagelearners in standards-based assessment has been described as providing“the leverage needed to raise expectations for English language learners,and the emphasis on higher level skills should improve the quality ofteaching provided to them” (Lachat, 1999, p.60), “feedback that will allowinstructional leaders to improve instructional programs” (Lacelle-Peterson & Rivera, 1994, p.64), and will ideally “…help students reachthe standards by (a) influencing what is taught and how it is taught (i.e.,„washback‟ to instruction), (b) providing data to guide instructionalmodifications, and (c) targeting resources to schools they are mostneeded” (Rivera & Vincent, 1997, p.336). In addition, Mehrens (2002)states that large-scale assessments have two major purposes: to drivereform and to gauge if reform policies have had an impact on studentlearning. These goals are especially important for English language
    • 150learners who often face socio-economic, cultural, and linguisticchallenges to academic achievement.Demographic Data Total respondents who answered the on-line questionnaire totaled55 – 35% are non-ESL certified teachers and 27% are ESL-certifiedteachers. The administrators accounted for the remaining 38%- 16% areAssistant Principals, 11% are Principals and 11% are ESL DistrictPersonnel. Conclusions The analysis of the quantitative data in Chapter IV led the researcherto draw the following conclusions: 1. The descriptive statistics showing the means of the 10th grade TAKS for ELA and Mathematics do not indicate improvement in performance despite the decrease in the percent of ELLs enrolled in a school. 2. All the obtained Pearson r correlation coefficients to determine whether there is a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006 were all significant at the 0.05 level, two-tailed. Both null hypotheses were rejected.
    • 151 3. The negative Pearson r correlation coefficients implied that as the percentage of ELLs in a school increased, performance on both English Language Arts and Mathematics decreased. 4. The linear regression equations may be used to predict outcomes in 10th grade TAKS tests in ELA and Mathematics using the percentage of ELLs enrolled in a school as the predictor variable. The analysis of the qualitative data in Chapter IV led the researcher todraw the following conclusions: 1. Texas Assessment of Knowledge and Skills (TAKS) were perceived by respondents as a tool to gauge knowledge in the core areas. 2. ELLs were expected to have at least average scores on TAKS. 3. There was a difference in the expected and actual results. Respondents observed dismal or failing performance of ELLS in actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. 4. Higher dropout rate and lower graduation rates of ELLs were problems encountered due to TAKS. 5. Respondents favored a different test for ELLs. 6. Respondents believed that interventions were needed to help ELLs perform better. Implications The research data gathered in the course of this study suggestedthat while there was a common perception that ELLs performed poorly
    • 152on high stakes testing, there was no unanimity among professionals inthe field of education regarding the viability of options that might beconsidered in addressing the low achievement level of ELLs. This was notnecessarily relevant, although it suggested that the appreciation of theproblem and its causes lent itself to biases and distortions depending onthe personal circumstance and perspective of those presenting theseoptions. It was clear from the study that schools needed to do thingsdifferently, if they expected ELLs to perform better on standardizedassessments. The major implications of the study were as follows: 1. The performance of schools in high stakes testing was affected by the size and proportion of ELLs taking the test. At the same time, ELLs were not evenly distributed across campuses. The performance of schools on standardized tests was influenced to a degree by the voluntary segregation in many districts of ethnic groups who speak English only as an adopted language. This was a phenomenon that was beyond the power of school districts to address, and that required wide coordination among various government agencies to develop an appropriate policy response. 2. The extended deferment of standardized tests administered in English to ELLS should be considered. The primary goal of these tests was to measure learning, that might be more accurately accomplished if the assessment was done in the language the student was most proficient. There was the expectation that the
    • 153 student will eventually be proficient in English as well. Since a second language is acquired in degrees, it might be reasonable to assume that ELLs would not readily have the same facility for English as a native speaker. Administering the test in English before the ELL student was ready for it would compromise the stated goal of measuring learning as accurately as possible.3. Learning is transmitted through communication. Due to the unique linguistic characteristic of ELLs, unique strategies, modifications, and instructions need to be used to maximize their capacity to learn concepts and skills. It is futile to assume that ELLs will learn the same way as native speakers of the English language. It follows that education professionals need the specialized training and support to be able to facilitate learning for ELLs.4. The Language Proficiency Assessment Committee (LPAC) in each campus needs to take a more active role in monitoring the progress of ELLs and devising specific plans to properly respond to the requirements designed specifically for ELLs. They need to undertake a regular evaluation of instruction and curriculum for ELLs and communicate findings and recommendations to all stakeholders – school administrators, teachers, parents and the ELLs.
    • 154 5. Interventions to improve the situation of ELLs should include specific action plans to devise a more intensive English program in schools and a continued emphasis on quality instruction employing strategies suggested by educational experts who have extensively researched on such courses of action. Recommendations for Further Study Based on the results of the study, the researcher recommends thefollowing concerns for further study: 1. A study to determine what additional supports are needed to ensure that English language learners will pass high-stakes tests. 2. A study to identify what data are needed to make fair high-stakes decisions about English language learners (like subject grades, samples of class work and recommendations of teachers and counselors). 3. A study to determine the reasons why English language learners scored lowest among student groups in the Texas Assessment of Knowledge and Skills tests in the core areas of English Language Arts and/or Mathematics. 4. A study to explore different approaches in school campuses regarding handling of English language learners in terms of instruction, curriculum and other pertinent or related aspects (such as some sort of evaluation - academic, social, financial, etc.)
    • 155 that may guide administrators and teachers to effectively handle English Language learners. 5. A study to determine the performance of 10th grade English language learners compared to non-English language learners and non-classified students based on the different objectives of the Texas Assessment of Knowledge and Skills tests in either or both Mathematics and English Language Arts. 6. A study to determine the impact of high stakes testing on English language learners as viewed by parents and students. 7. A study to explore different instruments to measure academic performance of English language learners. 8. A study to determine if there is significant difference between performance in the different core areas of English language learners belonging to different language groups. This study affirmed the expected outcome that a significantrelationship existed between the percentage of ELLs enrolled in a schooland the percentage of all students passing the 10 th grade TAKS tests inboth core areas of English Language Arts and Mathematics. Theregression analysis predicted that as the percentage of ELLs in a schoolincreased, the performance in the statewide, high-stakes testing in termsof all students passing the 10th grade TAKS tests decreased. Therespondents of the study considered the Texas Assessment of Knowledgeand Skills (TAKS) as a tool to gauge knowledge in the different core areas.
    • 156English language learners were expected to have at least average scoreson TAKS. There was a difference in the expected and actual results;respondents observed dismal or failing performance of ELLS in the actualresults in TAKS. This was evident by the high failure rate of ELLs in theirrespective schools. Higher dropout rate and lower graduation rate ofELLs were problems encountered due to TAKS. Respondents favored adifferent test for ELLs, possibly given at a later date after ELLs hadstudied in the country for at least several years. Respondents believedthat interventions were needed to help ELLs perform better. Both theschool and the home, together with the community have to be involved inpreparing ELLs to be better prepared for their present and future roles inthe American society.
    • REFERENCESAbedi, J. (2004). The No Child Left Behind Act and English Language Learners: Assessment and Accountability Issues. Educational Researcher, 33(1), 4-14.Abedi, J. (2004). Will You Explain the Question? Principal Leadership (High School Ed.), 4(7), 27-31.Abedi, J., Hofstetter, C. & Lord, C. (2004). Assessment Accommodations for English Language Learners: Implications for Policy-Based Empirical Research.Abedi, J. (2004). Inclusion of Students with Limited English Proficiency in NAEP: Classification and Measurement Issues. CSE Report 629 CRESST/ University of California, Los Angeles.Abedi, J. (2004). Will You Explain the Title? Principal Leadership (High School Ed.), 4(7), 27-31.Abedi, J., Leon, S., & Mirocha, J (2003). Impact of student language background on content-based performance: Analyses of extant data (CSE Tech. Rep. No. 603). Los Angeles: University of California, National Center for Research on Evaluation, Standards, and Student Testing.Abrams, L.M., & Madaus, G.F. (2003). The Lessons of High-Stakes Testing. Educational Leadership, 61(3), 31-35. 157
    • 158Airasian, P. W. (1988). Symbolic validation: The case of state-mandated, high-stakes testing. Educational Evaluation and Policy Archives, 10(4), 301-313.Albus, D., Thurlow, M. L., Liu, K. K., & Bielinski, J. (2005). Reading Test Performance of English Language Learners Using an English Dictionary. The Journal of Educational Research, 98(4), 245-254.Albus, D., Thurlow, M. L., & Liu, K. K. (2002). 1999-2000 Participation and Performance of English Language Learners Reported in Public State Documents and Web Sites (LEP Projects Report 3). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.Alderson, J. C., & Wall, D. (1993). Does Washback Exist? Applied Linquistics, 14(2), 115-129.Amrein, A. L., & Berliner, D. C. (2003). The Effects of High-Stakes Testing On Student Motivation and Learning. Educational Leadership, 60(5), 32-38.Anderson, M. E. (2001). Item Response Differences Among Spanish- Speaking ESL Students and Native English-Speaking Students on a Standardized Reading Test. Unpublished Manuscript.Anderson, M. E. (2004). Intended and Unintended Consequences of Statewide Testing for ESL Curriculum and Instruction. (UMI No. 3137152).
    • 159Anderson, M. E., Thurlow, M. L., Swierznin, B., Liu, K. K., & Bielinski, J. (2000). Bilingual Accommodations for Limited English Proficient Students on Statewide Reading Tests: Phase (No. 31 Minnesota report). Minneapolis, MN: University of Minnesota, National Center on Educational Outcomes.Artiles, A. J., Rueda, R., Salazar, J. J, & Higareda, I. (2005). Within Group Diversity in Minority Disproportionate Representation: English Language Learners in Urban City Schools. Exceptional Children, 71(3), 283-300.August, D., & Hakuta, K. (1997). Improving Schooling for Minority Children: A Research Agenda. Washington, D.C.: National Academy Press.Barro, S.M., & Kolstad, A. (1987). Who Drops Out of High School? Findings from High School and Beyond. Washington, D.C.: Center for Education Statistics, Office of Research and Improvement, U.S. Department of Education.Brennan, R. L. (2004). Revolutions and Evolutions in Current Educational Testing. Center for Advanced Studies in Measurement and Assessment: CASMA Research Report #6.California Department of Education. (2000). California demographics data. Retrieved February 24, 2003, from http://www.cde.ca.gov/demographics/
    • 160Center on Educational Policy (2005). From the capital to the classroom: Year 3 of the No Child Left Behind Act. Washington, DC: Author.Chamot, A. U., & O’Mailley. J.M. (1994). The CALLA Handbook: Implementing the Cognitive Academic Language Learning Approach. Reading, MA: Addison-Wesley.Cheng, L. (1999). Changing Assessment: Washback on Teacher Perceptions and Actions. Teaching and Teacher Education, 15, 253-271.Cheng, L. (2003). Looking at the Impact of a Public Examination Change on Secondary Classroom Teaching: A Hong Kong Case Study. Journal of Classroom Interaction, 38(1), 1-10.Cizek, G.J. (2003). When Teachers Cheat. Education Digest, 68(6), 28-31.Cobb, T. (2000). One Size Fits All? Francophone Learners and English Vocabulary Tests. Canadian Language Review, 57(2), 295-324.Collier, V.P. (1989). How Long? A Synthesis of Research on Academic Achievement in a Second Language. TESOL Quarterly, 23(3), 509- 531.Coltrane, B., (2002).English Language Learners and High-Stakes Tests: An Overview of the Issues. (November, 2002). Center for Applied Linguistics.Council of Chief State School Officers. (1992). Recommendations for Improving and Assessment and Monitoring of Students with
    • 161 Limited English Proficiency. Washington, DC: Council of Chief State School Officers.Creswell, J.W. (1998). Qualitative Inquiry and Research Design: Choosing Among Five Traditions. Thousand Oaks, CA: Sage.Davis, M. R. & Sacks, J. L. (2005). California, U.S. Department of Education strike deal on NCLB rules [Electronic version]. Education Week.Datnow, A., Borman, G.D., Stringfield, S., & Overman L.T. (2003). Comprehensive School Reform in Culturally and Linguistically Diverse Contexts: Implementation and Outcomes from a Four-Year Study. Educational Evaluation & Policy Analysis, 25(2), 143-170.D’Emilio, T. E. (2003). LEP Student Participation in Special Education: Over or Under-Representation? Paper Presented at the CCSSO Conference on Large-Scale Assessment, San Antonio, TexasDenzin, H. (1970). The research act: A Theoretical Introduction to Sociological Methods. New York: McGraw-Hill.Dexter, L.A. (1970). Elite and Specialized Interviewing. Evanston, IL: Northwestern University.Edwards, V. (Ed.). (2003, Jan. 9). Quality Counts 2003: If I can’t learn from you… (Education Week Special Report), 12(17). Bethesda, MD: Editorial Projects in Education.Erlandson, D., Harris, E., Skipper, B., & Allen, S. (1993). Doing Naturalistic Inquiry. Newbury Park: Sage.
    • 162Erpenbach, W.J., Forte-Fast, E., & Potts, A. (2003). Statewide educational accountability under NCLB (An Accountability Systems and Reporting State Collaborative on Assessment and Student Standards Paper). Washington, DC: Council of Chief State School Officers.Escamilla, K., Mahon, E., Riley-Bernal, H., & Rutledge, D. (2003). High- Stakes Testing, Latinos, and English Language Learners: Lessons from Colorado. Bilingual Research Journal, 27(1), 25-47.FairTest (2000).Flores, B.B., & Clark, E.R. (2003). Texas Voices Speak About High- Stakes Testing: Preservice Teachers, Teachers, and Students, Current Issues in Education (Vol.6).Fowler, F. J. (1984). Survey Research Methods. Beverly Hills: Sage.Fraenkel, J. & Wallen, N. (2003). How to Design and Evaluate Research In Education (5th ed.). New York, NY: McGraw-Hill.Freeman, Y.S., & Freeman, D., E. 2002). Closing the Achievement Gap: How to Reach Limited-Formal-Schooling and Long-Term English Learners. Portsmouth, NH: Heinemann.Garcia, G. E., & Pearson, P. D. (1994). Assessment and diversity. In L. Darling-Hammond (Ed.), Review of Research in Education (Vol. 20). Washington, DC: American Educational Research Association.Gewertz, C. (2005). Ed. Dept. allows Chicago to provide NCLB tutoring [Electronic version]. Education Week.
    • 163Goldhaber, D., & Hannaway, J. (2001). Accountability with a kicker: Observations on the Florida A+ Plan. Paper presented at the annual meeting of the Association of Public Policy and Management, Washington, DC.Goldschmidt, P. and Wang, J. (1999) "When Can Schools Affect Dropout Behavior: A Longitudinal Multilevel Analysis," American Educational Research Journal, vol. 36, no. 4, pp.715-738, Winter.Guilfoyle, C. (2006). “NCLB: Is There Life Beyond Testing?” Educational Leadership, vol. 64, no. 3, pp. 8-13, November 2006.Haladyna, T., Nolen, S., & Haas, N. (1991) Raising standardized achievement test scores and the origins of test score pollution. Educational Researcher, 20(5). 2-7Hammersley, M. (1992). What’s Wrong with Ethnography? London: Routledge.Haney, W. (2000). The Myth of the Texas Miracle in Education, Education Analysis Policy Archives (Vol.8).Harlow, A., & Jones, A. (2003, July). Why students answer TIMSS science test items the way they do. Paper presented at the annual conference of the Australian Science Education Research Association, Melbourne, Victoria, Australia.Herman, J., & Golan, S. (n.d.). Effects of standardized testing on teachers and learning (CSE Technical Report 334). Los Angeles:
    • 164 National Center for Research on Evaluation, Standards, and Student Testing.Herszenhorn, D. (2003, July 23). Basic skills forcing cuts in art classes. The New York Times, p. B1.Heubert, J. (2000).High-Stakes Testing: Opportunities and Threats. In Pines, M (ed.). The Continuing Challenge: Moving the Youth Agenda Forward (Policy Issues Monograph 00-02, Sar Levitan Center for Social Policy Studies. Baltimore, MD: Johns Hopkins University Press.Heubert, J.P. (2003). First, Do No Harm. Educational Leadership, 60(4), 26-30.Heubert, J., & Hauser, R. (Eds.) (1999). High stakes testing for tracking, promotion, and graduation. Washington, DC: National Academy Press.Hill, R.K., & DePascale, C. A. (2003). Reliability of no child left behind accountability designs. Educational Measurement: Issues and Practice, 22(3), 12-20.Hinde, E.R. (2003). The Tyranny of the Test: Elementary Teacher’s Conceptualizations of the Effects of State Standards and Mandated Tests on Their Practice, Current Issues in Education. (Vol.6).Hoffman, J. V., Assaf, L., & Paris, S. G. (2001). High Stakes Testing in Reading: Today in Texas, Tomorrow? The Reading Teacher.
    • 165Holmes, E. (1911). What is and what might be: A study of education in general and elementary in particular. London: Constable.Hood, L. (2003). Immigrant Students, Urban High Schools: the Challenge Continues. Revised 09/16/2003, from http://www.carnegie.orge/immigrantstudents.pdf.Horn, C. (2003). High-Stakes Testing and Students: Stopping or Perpetuating a Cycle of Failure? Theory Into Practice, 42(1), 30-41.Horn, C., Ramos, M., Blumer, I., & Madaus, G. (2000). Cut scores: Results may vary. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College.Isaac, S. & Michael, W. (1997). Handbook in Research and Evaluation for Education and the Behavioral Sciences (3rd ed.) San Diego, CA: EdiTs/Educational and Industrial Testing Services.Johnson, D. (2001). Performance Pentagon: Five Strategies to Help All Students Make the Grade. NASSP Bulletin, 85, 40-55.Jones, M., Jones, B., Hardin B., Chapman, L., Yarbough, T., & Davis, M. (1999). The impact of high-stakes testing on teachers and students in North Carolina. Phi Delta Kappan, 81(3), 199-203.Kaufman, P., Alt, M. N., & Chapman, C. D. (2001). Drop-Out Rates in the United States: 2000 (No.NCES 2002-114). Washington, DC: National Center on Educational Statistics.Kinder, A. L. (2002). Survey of the states’ limited English proficient students and available educational programs and services, 2000-
    • 166 2001 summary report. Washington, DC: National Clearinghouse for English Language Acquisition and Language instruction Educational Programs.Klein, S., Hamilton, L., McCaffrey, D., & Stecher, B. (2000). What do test scores in Texas tell us? Education Policy Analysis Archives, 8(49). [Online]. Available: http://epaa.asu.edu/epaa/v8n49Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996a). Perceived effects of the Kentucky Instructional results information system (KIRIS) (MR-792-PCT/FF). Santa Monica, CA: Rand.Koretz, D., Barron, S., Mitchell, K., & Keith, S. (1996b). Perceived effects of the Maryland school performance assessment program (CSE Technical Report 409). Los Angeles: National Center for Research on Evaluation, Standards, and Student Testing.Krathwohl, D. (1993). Methods of Educational and Social Science Research: An Integrated Approach. New York: Longman.Krueger, R.A., & Casey, M. A. (2000). Focus Groups: A Practical Guide for Applied Research (3rd ed.). Thousand Oaks, CA: Sage Publications.Lacelle-Peterson, M., & Rivera, C. (1994). Is it Real for all Kids? A Framework for Equitable Assessment Policies for English Language Learners. Harvard Educational Review, 64(1), 55-75.Lachat, M. A. (1999). What Policy Makers and School Administrators Need to Know About Assessment Reform for English Language
    • 167 Learners. Northeast and Islands Regional Educational Laboratory at Brown University. Providence.Lane, S., & Stone, C. A. (2002). Strategies for Examining the Consequences of Assessment and Accountability Programs. Educational Measurement: Issues into Practice, 21(1), 23-30.Lewis, A. (2002). A Horse Called NCLB. Phi Delta Kappan, 84(3), 179-180.Linn, R. (1998). Assessments and Accountability (CSE Technical Report 490). Boulder, CO: CRESST/University of Colorado at Boulder.Linn, R. L. (2000). Assessment and Accountability. Educational Researcher, 29(2), 14-16.Linn, R. L., Baker, E. L., & Herman, J. L. (2002). Minimum group size for measuring adequate yearly progress: The CRESST line. Los Angeles: University of California, Center for the Study of Evaluation/National Center for Research on Evaluation, Standards, and Student Testing.Liu, K. & Thurlow, M. (1999). Limited English Proficient Students’ Participation and Performance on Statewide Assessments: Minnesota Basic Standards Reading and Math, 1996-1998. National Center on Educational Outcomes.Madaus, G. (1988). The influence of testing on the curriculum. In L. Tanner (Ed.), Critical issues in curriculum (pp. 83-121). Chicago: University of Chicago Press.
    • 168Madaus, G. (1991, January). The effects of important tests on students: Implications for a national examination or system of examinations. Paper prepared for the American Educational Research Association Invitational Conference on Accountability as a State Reform Instrument, Washington, DC.Madaus, G., & O’Dwyer, L. (1999). A Short History of Performance Assessment. Lessons learned. Phi Delta Kappan, 80(9), 688-695.Major, R.C., Fitzmaurice, S.M., Bunta, F., & Balasubramanian, C. (2005). Testing the Effects of Regional, Ethnic, and International Dialects of English on Listening Comprehension. Language Learning 55 (1) 37-69.Marshak, D. (2003). No Child Left Behind: A Foolish Race Into the Past.Phi Delta Kappan, 85(3), 229-231.Marshall, C. (1990). Goodness Criteria: Are They Objective or Judgment Calls? In E. G. Guba (Ed.), The Paradigm Dialog (pp. 188-197). Thousand Oaks, CA: Sage.Maxwell, J. (2005). Qualitative Research Design. An Interactive Approach (2nd ed.). Thousand Oaks, CA: Sage Publications.McLaughlin, M.W., & Shepard, L.A. (1995). Improving Education Through Standards-Based Reform. Stanford, CA: The National Academy of Education.McMillan, J., Myran, S., & Workman, D. (1999, April 19-23). The Impact of Mandated Statewide Testing on Teachers’ Classroom
    • 169 Assessment and Instructional Practices. Paper presented at the annual meeting of the American Educational Research Association, Montreal, Quebec, Canada.McNeil, L.M. (2000). Contradictions of school reform: Educational costs of standardized testing. New York: Routledge.Mehrens, W. A. (1998). Consequences of assessment: what is the evidence? Vice Presidential Address for Division D, annual meeting of the American Educational Research Association, San Diego.Menken, K. (2000).What Are the Critical Issues in Wide-Scale Assessment of English Language Learners? Framing Effective Practice, 23-27.Merriam, S. B. (1998). Qualitative Research and Case Study Applications in Education, San Francisco, CA: Jossey-Bass.Miller & Fredericks (2000). Research on High-Stakes Tests and Instruction. Retrieved October 9, 2005 from http://vnweb.hwwilsonweb.com/hww/results/results_single_fullte xt.jhtml.Murnane, R. & Levy, F. (1996). Teaching the New Basic Skills. New York: The Free Press.Murnane, Richard J., John B. Willett, Yves Duhaldeborde, and John H. Tyler. 2000. How important are the cognitive skills of teenagers in predicting subsequent earnings? Journal of Policy Analysis and Management 19 (4) 547-568.
    • 170Myers, K. (2003, July 16). A dream denied: Aspiring chef rethinks her future as Falmouth school board bows to state pressure on MCAS. Cape Cod Times. Available: www.capecodonline.com.National Center for Education Statistics. (2002). Public school student, staff, and graduate counts by state: School year 2000-01 (NCES Publication 2002-348). Washington, DC: Author.National Clearinghouse for Bilingual Education. (1997). High-Stakes Assessment: A Research Agenda for English Language Learners. Retrieved June 21, 1999, from http://www.ncbe/gwu.edu/ncbepubs/reports/highstakes.htm#Te sting.National Commission of Excellence in Education. (1983). A Nation at Risk: The Imperative for Educational Reform: a Report to the Nation and the Secretary of Education, United States Department of Education. Washington, D.C.: The Commission: [Supt. of Docs., U.S. G.P.O. distributor].No Child Left behind Act of 2001, 20 U.S.C. 6301 et.seq. (2002).OByrne, B. (2001). Needed: A Compass to Navigate the Multilingual English Classroom. Journal of Adolescent & Adult Literacy 44 (5) 440-9.Olson, L. (2005). Defying predictions, state trends prove mixed on schools making NCLB targets [Electronic version]. Education Week.
    • 171Orfield, G. & Kornhaber, M. L. (2001). Raising standards or raising barriers? Washington, DC: The Century Foundation Press.Patton, M. Q. (1990). Qualitative Evaluation and Research Methods (2nd ed.). Thousand Oaks, CA: Sage Publications.Pedulla, J., Abrams, L., Madaus, G., Russell, M., Ramos, M., & Miao, J (2003). Perceived effects of state-mandated testing programs on teaching and learning: Findings from a national survey of teachers. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College.Popham, J. W. (2003). The Seductive Allure of Data. Educational Leadership, 60(5), 48-51.Reyes, P. & Fletcher, C. (2003). Successful Migrant Students: The Case of Mathematics. Journal of Curriculum and Supervision, 18(4), 306- 333.Rex, L. A. & Nelson, M. C. (2004). How Teachers’ Professional Identities Position High-Stakes Test Preparation in Their Classrooms. Teachers College Record, 106(6), 1288-1331.Rhoades, K., & Madaus, G. (2003). Errors in standardized tests: A systemic problem. Chestnut Hill, MA: National Board on Educational Testing and Public Policy, Boston College.Rivera, C., & Stansfield, C. (1998). Leveling the Playing Field for English Language Learners: Increasing the Participation in State and Local Assessments Through Accommodations. In R. Brandt (Ed.),
    • 172 Assessing Student Learning: New Rules, New Realities (pp. 65-92). Arlington, VA: Educational Research Service.Rivera, C., Stansfield, C., Scialdone, L., & Sharkey, M. (2000). An analysis of state policies for the inclusion and accommodation of ELLs in state assessment programs during 1998 – 1999 (Executive Summary). Washington, DC: The George Washington University, Center for Equity and Excellence in Education.Rivera, C., & Vincent. (1997), High School Graduation Testing: Policies and Practices in the Assessment of English Language Learners. Educational Assessment, 4(4), 335-355.Rose, L.C. & Gallup, A.M. (2003). The 35 th Annual Phi Delta Kappa/ Gallop Poll of the Public’s Attitudes Toward the Public Schools. Phi Delta Kappan, 85.1, 41-56.Rosenbusch, M. H. (2005). The No Child Left Behind Act and Teaching and Learning Languages in U.S. Schools. The Modern Language Journal, 89(2), 250-261.Russell, M., & Abrams, L. (in press). Instructional uses of computers for writing: The impact of state testing programs. Teachers College Record.Sheldon K. and Biddle, B. (1998). Standards, accountability and school reform: Perils and pitfalls. Teachers College Record 100 (1): 164- 180.Sirkin, R. M. (2006). Statistics for the Social Sciences. (3rd ed.). Thousand
    • 173 Oaks, CA: Sage Publications.Solano-Flores, G., & Trumbull, E. (2003). Examining Language in Context: The Need for New Research and Practice Paradigms in the Testing of English Language Learners. Educational Researcher, 32(2), 3-13.Spellings, M. (2005). Decision Letter on Request to Amend Virginia accountability plan. Retrieved November 5, 2005 from http://www.ed.gov/print/admins/lead/account/letters/acva5.ht ml.Stake, R. (1999). The Goods on American Education. Phi Delta Kappan, 668-672.Stecher, B., Barron, S., Chun, T., & Ross, K. (2000). The effects of the Washington state education reform on Standards and Student Testing.Sum, A. (1999). Literacy in the Labor Force. Washington, DC: National Center for Education Statistics.Taylor, G., Shepard, L., & Kinner, F. (2003). A Survey of Teachers’ Perspectives on High-Stakes Testing in Colorado: What Gets Taught, What Gets Lost (CSE Technical Report No. 588). Santa Cruz, CA and Los Angeles, CA: Center for Research on Evaluation, Diversity, and Excellence and CRESST.Technical Digest, 2003 – 2004
    • 174Texas Assessments of Knowledge and Skills (TAKS), (2005). Data Resources and Research. Texas Education Agency (TEA) from http://www.tea.state.tx.us/data.html.Thurlow, M. L., Nelson, J. R., Teelucksingh, E., & Ysseldyke, J. E. (2000). Where’s Waldo? A Third Search for Students with Disabilities in State Accountability Reports (Technical Report No.25). Minneapolis, MN: National Center on Educational Outcomes.Thompson, G. L. The Real Deal on Bilingual Education: Former Language-Minority Students Discuss Effective and Ineffective Instructional Practices. Educational Horizons, 78(20, 80-92.U. S. Department of Education. (2002). Elementary and Secondary Education. Title IX. General Provisions. Part A, Section 9101. Definitions. [11] Core Academic Subjects. Washington, D.C.: U.S. Dept. of Education.U.S. Department of Education, Office of the Undersecretary, Planning and Evaluation Service. (1999). Promising results, continuing challenges: The final report of the national assessment of Title I. Washington, DC: AuthorU.S. Department of Education. (1995). The Improving America’s Schools Act of 1994: Reauthorization of the Elementary and Secondary Education Act. Washington, D.C.: U.S. Dept. of Education.Van Dalen, D.B. (1979). Understanding Educational Research (4th ed.). New York: McGraw-Hill.
    • 175Wall, M. E. (2004). A Case Study of Secondary School Efforts Toward English Language Learner Success in a Standards-Based Reform System.Wang, M. & Koda, K. (2005). Commonalities and Differences in Word Identification Skills among English Second Language Learners. Language Learning, 55 (1) 71-98.Weitzman, E.A. (2000). Software and Qualitative Research. In N.K. Denizin & Y.S. Lincoln (Eds.), The Handbook of Qualitative Research (2nd ed., pp. 803-820). Thousand Oaks, CA: Sage.Whoriskey, P. (2006).Political backlash builds over high-stakes testing. Public support wanes for tests seen as punitive. Washington Post Company.Winograd, M. (2002). Equity: A Prerequisite for Reform. Principal Leadership, 2(8), 42-47.Wright, W. E., (2006).A Catch-22 for Language Learners. Educational Leadership. November 2006/Volume 64/Number 3.Zehler, A. M., Fleischman, H. L., Hopstock, P. J., Pendzick, M. L., & Stephenson, T. G. (2003). Descriptive Study of Services to LEP Students and LEP Students with Disabilities. (No. 4 Special Topic Report: Findings on Special Education LEP. students). Arlington, Va.: Development Associates, Inc.
    • 176Zehler, A. M., Hopstock, P. J., Fleischman, H. L., & Greniuk, C. (1994). An Examination of Assessment of Limited English Proficient Students. Special Issues Analysis Center,Task Order Report. Arlington, VA: Development Associates.
    • APPENDICES 177
    • 178APPENDIX A IRB
    • 179 PRAIRIE VIEW A&M UNIVERSITY The Texas A&M University System P. O. Box 4149 Prairie View, Texas 77446-4149Office of the Vice President v. 936.857.4494Research and Development f. 936. 857.2255 March 21, 2006TO: Mr. Arthur L. Petterway, Principal Investigator Doctoral Student, Education Leadership and Counseling Dr. Ben DeSpain, EDLC – Faculty AdvisorFROM: Marcia C. Shelton, Compliance Officer, Regulatory Research Institutional Review BoardSUBJECT: IRB Protocol Review – Protocol StatusTitle: A Mixed Mthods Analysis of the Impact of High Stakes Testing on English LanguageLearners in Major Urban High Schools in TexasProtocol Number: 200-103Review Category: Full Review - (primary reviewer –L. Myers)Approval Date: February 13, 2006The approval determination was based on the following Code of Federal Regulations (CFR)46.101(b) (2). Research involving the use of educational tests (cognitive, diagnostic,aptitude, achievement), survey procedures, interview procedures or observation of publicbehavior, unless:(i) information obtained is recorded in such a manner that human subjects can beidentified, directly or through identifiers linked to the subjects; and (ii) any disclosure ofthe human subjects responses outside the research could reasonably place the subjects atrisk of criminal or civil liability or be damaging to the subjects financial standing,employability, or reputation.Remarks:The Institutional Review Board – Human Subjects in Research, Prairie View A&M Universityhas reviewed and approved the above referenced protocol. Your study has been approved forone year –February 13, 2006- February 12, 2007. As the principal investigator of this study, youassume the following responsibilities:Renewal: Your protocol must be re-approved each year in order to continue the research. Youmust also complete the proper renewal forms in order to continue the study after the initialapproval period.Adverse events: Any adverse events or reactions must be reported to the IRB immediately.Amendments: Any changes to the protocol, such as procedures, consent/assent forms, addition ofsubjects, or study design must be reported to and approved by the IRB.Completion: When the study is complete, you must notify the IRB office and complete therequired forms.
    • 180Date: 14-Jan-2006 00:33:10 -0600From: <od_crt@mail.cc.nih.gov>To: <apetterway@pvamu.edu>Subject: Exam Confirmation - Clinical Research TrainingRegistration Summary:01/14/2006Arthur L Petterway Ph.DPrairie View A&M UniversityEducational Leadership & CounselingP. O. Box 519n/aMSCPrairie View TX 77446-0519United StatesTel: (713)924-1622 Fax: (713)924-1619E-mail: apetterway@pvamu.eduThis e-mail is to verify that you successfully completed the NIH Clinical ResearchTraining course.You answered 20 out of a total of 25 questions for a final grade of 80%.If you are an NIH principal investigator, you have fulfilled the Training and EducationStandard issued by the NIH for conducting clinical research within the intramuralresearch program.Please print this e-mail and retain for your records.
    • 181CITI Course in The Protection of Human Research Subjects Sunday, February 19, 2006 CITI Course Completion Record for Arthur PetterwayTo whom it may concern:On 1/15/2006, Arthur Petterway (username=apetterw) completed all CITIProgram requirements for the Basic CITI Course in The Protection of HumanResearch Subjects.Learner Institution: Texas A&M UniversityLearner Group: Group 2.Learner Group Description: Social and Behavioral Research Investigators andKey PersonnelContact Information: Gender: Male Which course do you plan to take?: Social & Behavioral Investigator Course Only Role in human subjects research: Principal Investigator Mailing Address: 5300 N. Braeswood Blvd #247 Houston Texas 77096 USA Email: apetterway@pvamu.edu Office Phone: (713)924-1622 Home Phone: (713)498-8667
    • 182 DateThe Required Modules for Group 2. are: completedIntroduction 01/15/06History and Ethical Principles - SBR 01/15/06Defining Research with Human Subjects - SBR 01/15/06The Regulations and The Social and Behavioral Sciences - 01/15/06SBRAssessing Risk in Social and Behavioral Sciences - SBR 01/15/06Informed Consent - SBR 01/15/06Privacy and Confidentiality - SBR 01/15/06Research with Prisoners - SBR 01/15/06Research with Children - SBR 01/15/06Research in Public Elementary and Secondary Schools - SBR 01/15/06International Research - SBR 01/15/06Internet Research - SBR 01/15/06Conflicts of Interest in Research Involving Human Subjects 01/15/06Texas A&M University 01/15/06 DateAdditional optional modules completed: completedFor this Completion Report to be valid, the learner listed above must beaffiliated with a CITI participating institution. Falsified information andunauthorized use of the CITI course site is unethical, and may be consideredscientific misconduct by your institution.Paul Braunschweiger Ph.D.Professor, University of MiamiDirector Office of Research EducationCITI Course Coordinator
    • 183 APPENDIX BCONSENT FORM
    • 184 CONSENT FORM A MIXED METHOD ANALYSIS OF THE IMPACT OF HIGH STAKESTESTING ON ENGLISH LANGUAGE LEARNERS IN MAJOR URBAN HIGH SCHOOLS IN TEXASI have been asked to participate in a research study. This study isintended to explore what certified English as a Second Language (ESL)teachers, non-certified ESL teachers who teach English LanguageLearners (ELLs), campus administrators , and district ESL personnelview as the impact that high stakes assessments have on ELLs, ESLcurriculum and instruction , and what they observe as occurring. I wasselected to be a possible participant because I am either a campusadministrator, a teacher handling ELLs or a district ESL coordinator ofthe selected major urban high schools in Texas. A total of 118 peoplehave been asked to participate in this study. The purpose of this study isto determine the views and opinions of campus administrators, teachershandling ELLs, and district ESL coordinators regarding the impact ofstatewide testing, specifically Texas Assessment of Knowledge and Skills(TAKS), on the curriculum and instruction of this special group ofstudents.If I agree to be in this study, I will be asked to answer an on-linequestionnaire. I may also form part of the focus group which will havegroup interviews verifying and validating my views and opinionsregarding the open-ended questions. If I am a principal or a district ESLcoordinator my interview will be one-on-one. The proceedings will betranscribed so that the researcher can go back to sections for betterunderstanding. I may volunteer another knowledgeable respondent inorder that the succeeding interviews will be more productive ormeaningful. This study will only take at least two interviews after the on-line questionnaire has been answered. The risks associated with thisstudy are almost non-existent. The benefits of participation are mycontributions to the betterment of the ESL curriculum, instruction andtesting.I will receive no compensation for my participation in this researchstudy. A simple ‘thank you’ note will suffice.This study is confidential since the data will be dealt with only by theresearcher and transcripts will be kept in a safe storage. The records ofthis study will be kept private. No identifiers linking me to the study willbe included in any sort of report that might be published. Date _____________ Initial________________ Page 1 of 2
    • 185Research records will be stored securely and only the researcher, ArthurL. Petterway, will have access to the records. For the transcripts, onlyArthur L. Petterway will also have access to the information containedtherein. My decision whether or not to participate will not affect mycurrent or future relations with Prairie View A & M University. If I decideto participate, I am free to refuse to answer any of the questions thatmay make me uncomfortable. I can withdraw at any time without myrelations with the University, job, benefits, etc., being affected. I cancontact Arthur L. Petterway at (713)748-8303 or apetterway@pvamu.eduand Dr. Robert Marshall, at (936)857-4127 Marshall@pvamu.edu withany questions about this study.This research study has been reviewed by the Institutional Review Board-Human Subjects in Research, Prairie View A & M University. Forresearch- related problems or questions regarding subjects’ rights, I cancontact the Institutional Review Board through Ms. Marcia C. Shelton,Research Compliance Officer, Anderson Hall Room 311, PO Box 4149,Prairie View, TX 77446, at 936.857.2541 and at mashelton@pvamu.edu.I have read the above information. I have asked questions and havereceived answers to my satisfaction. I have been given a copy of thisconsent document for my records. By signing this document, I consent toparticipate in the study.Signature of the Subject: ______________________________ Date: __________Signature of Investigator: ______________________________ Date: __________Original – ResearcherCopy – Participant Date _____________ Initial________________ Page 2 of 2
    • 186 APPENDIX CINTERVIEW QUESTIONS
    • 187 INTERVIEW /FOCUS GROUP QUESTIONS Please help us serve English Languages Learners (ELLs) moreeffectively by taking a few moments to answer these interviewquestions. Confidentiality will be maintained throughout this process, asonly I will have access to the data. Data and information will be kept in asafe home vault for a period of seven years, after which time they will bedestroyed. Summarized data will be published in my dissertation. Yourparticipation is greatly appreciated, as you will be making a significantcontribution to the English as a Second Language (ESL) curriculum,instruction and testing. 1. Describe your experience with English Language Learners. 2. In your estimation, how many English Language Learners graduate each year from your school? 3. Do you think this number has increased or decreased in recent years? Why? 4. Describe your school’s support system for former English Language Learners. 5. What is your understanding of the Texas Comprehensive Assessment Plan as it relates to ELLs? 6. Should the English Language Learner be held to a separate standard of promotion or to the same standards as the regular population? Why?
    • 188 7. With respect to your school, how do the promotion standards affect your English Language Learner population? What will it take to facilitate the English Language Learners’ success in meeting graduation standards? 8. What concerns need to be addressed before administrating TAKS to the English Language Learner? Who should be responsible in resolving these concerns? 9. Why should English Language Learners be held/or not held liable or accountable to the same standards or requirements?10. What do you think are the intended consequences of the statewide accountability test (TAKS), specifically in terms of ESL curriculum and instruction?11. Has the ESL program implemented in schools failed/or served its purpose or the state’s standardized assessment?12.How would you rate the success of the ESL program in regards to standardized assessment?13. To what extent do the consequences of a statewide assessment affect the ELL student, educator and district?14. In what ways, if any, has the curriculum for ESL students changed as a result of their participation in the assessment?15.What is your opinion of the cause of the low performing scores of ELL students?
    • 18916.What do you need to do to adequately prepare these students for success with standardized assessments and graduation standards?17. What are your recommendations for the future testing of the English Language Learner?18.Given that the criteria being used are high-stakes, what additional supports are needed to ensure that ELLs will be able to meet them?19.How do alternative assessments (e.g., Spanish language exams) compare to mainstream assessments?20.When is the use of native language assessments appropriate?21.How does the placement of accommodations impact comparability with mainstream student performance?22.Do wide-scale tests with the permitted accommodations fully assess English Language Learners’ knowledge and abilities or does the system need to be fully redesigned such that the needs of these students are addressed in the development of the assessments?23.Do you think other data collection methods, such as portfolios or other performance assessments, would yield more accurate results with regard to ELLs than traditional assessments?24.What sorts of information is needed to make fair high-stakes decisions about ELLs (e.g., grades, classroom performance, an array of samples of student work, teacher recommendations)?
    • 19025.What would be the most beneficial system(s) of accountability to ensure that these students are making progress in what they know and can do in important content areas?26.What supports are necessary to aid states and districts in their alignment of assessments, standards, curricula, and instruction?TAKS = Texas Assessment of Knowledge and Skills
    • 191 APPENDIX DON-LINE/HARD COPY QUESTIONNAIRE
    • 192A MIXED METHODS ANALYSIS OF THE IMPACT OFHIGH STAKES TESTING ON ENGLISH LANGUAGELEARNERS IN MAJOR URBAN HIGH SCHOOLS INTEXASPlease help us serve English Languages Learners(ELLs) more effectively by taking a few moments to fill out thisquestionnaire. The results will be returned to usautomatically via the web. Confidentiality will be maintainedthroughout this process, as only I will have access to thedata. A random numeric code will be electronically generatedand assigned to each qualitative questionnaire. Data andinformation will be kept in a safe home vault for a period ofseven years, after which time they will be destroyed.Summarized data will be published in my dissertation. Yourparticipation is greatly appreciated, as you will be making asignificant contribution to the English as a Second Language(ESL) curriculum, instruction and testing.1.) GENDER MALE FEMALE2.) AGE 21-30 31-40 41-50 51-60 61+3.) CURRENT POSITION Principal Assistant Principal
    • 193 ESL Certified Teacher NON-ESL Certified Teacher (who teach ELLs) District ESL Personnel4.) HIGHEST DEGREE EARNED Bachelors Masters Doctorate5.) What certifications do you hold?6.) YEARS OF EXPERIENCE IN PUBLIC EDUCATION 1-5 6-10 11-20 21+7.) What are the anticipated results of the statewide testing,specifically, English Language Arts (ELA) and Mathematicsportion of the Texas Assessment of Knowledge and Skills(TAKS), on English Language Learners?
    • 1948.) What are the actual results of the statewide testing,specifically, ELA and Mathematics portion of the TexasAssessment of Knowledge and Skills (TAKS), on EnglishLanguage Learners?9.) Why is the Texas Assessment of Knowledge and Skills(TAKS) given as a statewide test?10.) What are the intended consequences of this statewidetesting as it relates to English Language Learners?11.) What has happened because of TAKS as it relates toEnglish Language Learners?12.) What problems have occurred, if any, to the EnglishLanguage Learners because of TAKS?
    • 195 13.) What changes have occurred as a result of statewide testing as it relates to English Language Learners? 14.) What general recommendations would you suggest to improve the overall performance of English Language Learners on statewide testing ? 15.) Based on your recommendations above, which will be of greatest value for ELLs success on statewide testing? Thank you for taking the time to fill out this questionnaire. If you need to contact us - you can click on the following email apetterw@houstonisd.org Submit Form Reset Form-1
    • 196 APPENDIX ELETTER TO PARTICIPANTS
    • 197 LETTER TO PARTICIPANTSArthur L. Petterway 05/12/20065300 N. Braeswood Blvd. # 247Houston, Texas 77096Dear Sir or Madam, I am presently in the Ph.D. in Education Leadership program atPrairie View A & M University. I am currently conducting a dissertationresearch on high-stakes testing and English language learners. Mydissertation topic is “A Mixed Methods Analysis of the Impact of High-Stakes Testing on English Language Learners in Major Urban HighSchools in Texas”. I realize how busy you are in meeting the challenges ofyour work but I hope that you will take time to complete an on-linequalitative questionnaire that I have prepared to gather research data.You can access the qualitative questionnaire atapetterway@houstonisd.org. The questionnaire will include a major question on what are theanticipated and observed consequences of the statewide testing,specifically, Texas Assessments of Knowledge an Skills (TAKS), onEnglish as a Second Language (ESL) curriculum and instruction asviewed by certified ESL teachers, non-certified ESL teachers who teachEnglish Language Learners (ELLs), school administrators, and districtESL personnel. Confidentiality will be maintained throughout this process, as onlyI will have access to the data. A random numeric code will beelectronically generated and assigned to each qualitative questionnaire.Data and information will be kept in a safe home vault for a period ofseven years, after which time they will be destroyed. Summarized datawill be published in my dissertation. Your participation is greatlyappreciated, as you will be making a significant contribution to the ESLcurriculum, instruction and testing. Should you have any questions or comments, please feel free tocontact me at (713)748-8303 or apetterway@pvamu.edu. You mayContact my dissertation chair, Dr. Robert Marshall, at (936)857-4127 orMarshall@pvamu.edu. Once again, I appreciate your attention to thismatter and look forward to your favorable response.Sincerely,Arthur L. PetterwayPh.D. student – Prairie View A & M University
    • 198 APPENDIX FREQUEST FOR EXTANT DATA FROM TEXAS EDUCATION AGENCY (TEA)
    • 199 HOUSTON INDEPENDENT SCHOOL DISTRICT STEPHEN F. AUSTIN SENIOR HIGH SCHOOL 1700 DUMBLE HOUSTON, TEXAS 77023-3195 PH: (713) 924-1600 FAX: (713) 924-1619 ARTHUR L. PETTERWAY Assistant PrincipalDear Mr. Rod Rowell,As per our telephone conversation on July 3, 2006, I am submitting my request for extant data to TEA.Requestor Name: Arthur L. PetterwayCompany Name: PhD Doctoral Student at Prairie View A&M UniversityAddress: 5300 N. Braeswood Blvd. #247City/State/Zip: Houston, Texas 77096Telephone: Office-(713)924-1600; Home-(713)748-8303; Cell-(832)693-2809Fax Number: (713)924-1619Requestor Email Address: apetterw@houstonisd.orgBrief Summary of Request: I am currently working on my dissertation, A Mixed Methods Analysis of theImpact of High Stakes Testing on English Language Learners in Major Urban High schools in Texas, atPrairie View A&M University and need your help. This is a Public Information Request to please provide mewith the 2002-2003; 2003-2004; 2004-2005; 2005-2006 TAKS Summary Report-(ALL) for 10th gradeEnglish Language Arts and Mathematics of all of the high schools in the following school districts:015907 SAN ANTONIO ISD015910 NORTH EAST ISD015915 NORTHSIDE ISD057905 DALLAS ISD071902 EL PASO ISD071905 YSLETA ISD101912 HOUSTON ISD220901 ARLINGTON ISD220905 FORT WORTH ISD227901 AUSTIN ISD.Your immediate attention to this matter will be greatly appreciated. If you have any questions, please dont hesitate tolet me know.Sincerely,Arthur L. PetterwayAssistant Principal Home of the Mighty Mustangs
    • 200From: Woli, Urbe [mailto:Urbe.Woli@tea.state.tx.us]Sent: Tue 7/11/2006 3:23 PMTo: Petterway, Arthur LCc: PIR; Eaton, Jennifer; Woli, UrbeSubject: PIR # 6541 - Grade 10 Campus TAKS Summary DataMr. Arthur Petterway,The 2003 – 2005 TAKS campus summary data you are requesting (see attached PDFdocument) are available for download at no cost from our website athttp://www.tea.state.tx.us/student.assessment/reporting/taksagg/index.html. The 2006TAKS campus summary data will be posted at the same location by the end of August,2006. Due to the large size of the downloadable data files, you will need to use SAS orSPSS to process the data files.Please let us know if you prefer that we generate the data for you. To enable us provideaccurate data; please list all the statistics you need in the data. Contact me at (512) 463-9536 to discuss your data needs.Thank you.Urbe WoliStudent Assessment DivisionTexas Education AgencyTel: (512) 463-9536
    • VITA ARTHUR L. PETTERWAY 5300 N. Braeswood Blvd, #247 Houston, Texas 77096EDUCATIONAL HISTORY Prairie View A&M University, Prairie View, Texas, Ph. D. in Education Leadership, Expected Graduation Date August, 2007 Prairie View A&M University, Prairie View, Texas, M.Ed. in Education Administration, August, 1999 Dillard University, New Orleans, Louisiana B.A. in Biology, June, 1969CERTIFICATIONS Administrator - Principal Teacher - Elementary Mathematics (Grades 1-8) Teacher - Elementary Self-Contained (Grades 1-8)EMPLOYMENT HISTORY 2002 - 2007 Assistant Principal, Houston ISD 1996 - 2002 Math Department Chair, Houston ISD 1994 - 1996 Middle School Math Teacher, Houston ISD 1991 - 2006 Upward Bound Coordinator, UH/DT 1989 - 1991 President/CEO, Way Refining, Houston 1978 - 1989 Executive Vice President, Manufacturing, Houston 1969 - 1978 Manufacturing Supervisor, General Motors, Dayton, Ohio 201