Your SlideShare is downloading. ×
Dr. Arthur L. Petter, PhD Dissertation Defense, Dr. William Allan Kritsonis, Dissertation Committee Member
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Dr. Arthur L. Petter, PhD Dissertation Defense, Dr. William Allan Kritsonis, Dissertation Committee Member

843
views

Published on

Dr. William Allan Kritsonis, Dissertation Committee for Dr. Arthur L. Petterway, PhD Program in Educational Leadership, Prairie View A&M University/Member of the Texas A&M University System.

Dr. William Allan Kritsonis, Dissertation Committee for Dr. Arthur L. Petterway, PhD Program in Educational Leadership, Prairie View A&M University/Member of the Texas A&M University System.

Published in: Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
843
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Good afternoon! I am privileged to present my dissertation on: “ A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas”.
  • This study used the explanatory design of mixed methods analysis to : 1.Display the performance of English Language Learners in the high stakes testing called the Texas Assessment of Knowledge and Skills or TAKS; and, 2.Determine the impact of this same high stakes testing on English Language Learners as viewed by district personnel, school administrators and teachers.
  • The purpose of this study was to determine the impact of high-stakes testing on English language learners.
  • The following research question guided the quantitative dimension of the study: Is there a relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005 and 2006?
  • Two null hypotheses were formulated in line with the quantitative research question; the first null hypothesis dealt with English Language Arts: 1.There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in English Language Arts given in 2003, 2004, 2005 and 2006.
  • The second null hypothesis dealt with Mathematics : 2. There is no statistically significant relationship between the percentage of English language learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in Mathematics given in 2003, 2004, 2005 and 2006.
  • The subjects of the study included the following: I. Quantitative – 173 Urban High Schools in Texas II. Qualitative - Six principals responded, together with 9 assistant principals, 6 ESL district personnel, 15 ESL certified teachers and 19 non-ESL certified teachers – a total of 55 out of 98 purposive respondents, for a response rate of 56%.
  • Quantitative data were accessed and retrieved from the TEA website regarding the major urban high schools in Texas. Data were organized for computations utilizing the SPSS software package, Version 14.0. The on-line, open-ended questionnaire provided one of the bases for the qualitative data. The triangulation method included categorizing the responses to the online, open-ended questionnaire into emergent themes, interviewing the focus groups of teachers and assistant principals and one- on-one interviews with the principals and district ESL personnel.
  • For the quantitative portion of the study: The Descriptive Statistics were used to show percentage of English language learners during the four years under study and the corresponding performance of 10 th grade students (in terms of means or averages) in the TAKS test . Pearson r Correlation Coefficients were then calculated to determine if there was a significant relationship between the percentage of Ells enrolled in a school and the percentage of all 10 th grade students passing the TAKS test in the core areas of English Language Arts and Mathematics. The simple regression analysis yielded the linear equations which can be used to predict the outcome variable of percentage of all 10 th grade students passing in English Language Arts/Mathematics given the predictor variable of percentage of Ells enrolled in a school .
  • For the regression analysis, the percentage of ELLs enrolled in a school was considered as the predictor variable. The outcome variable was the percentage of all students passing the 10 th grade TAKS test in ELA and Mathematics.
  • Although the descriptive statistics did not show that when the percentage of ELLs enrolled in a school decreased there was an expected improvement in performance of all students passing the 10 th grade TAKS, it was in the computation of Pearson r correlation that results showed that there was significant relationship between the percentage of ELLs enrolled in a school and the percentage of all students passing in the 10 th grade TAKS in ELA and Mathematics. All of the obtained Pearson r Correlation coefficients were significant at the 0.05 level (two-tailed).
  • The qualitative research question focused on this main concern: What are the anticipated and observed consequences of the statewide testing specifically TAKS, on ESL curriculum and instruction as viewed by ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
  • An online open-ended questionnaire was developed from the existing review of literature. Two schools not included in the study were selected for the subjects’ expert opinions while at the same time establishing the trustworthiness of the instrument. Upon completion of the pilot study, the instrument was determined to be trustworthy. (In order to build trustworthiness, four criteria were considered: (1) CREDIBILITY, so that findings are believable and convincing ; (2) TRANSFERABILITY, so that findings can be applied to other settings; (3) DEPENDABILITY, so that findings are consistent with those of other similar investigations; and, (4) CONFIRMABILITY, to ensure that both the processes and the product are auditable (Isaac and Michael, 1995).)
  • The major research question for the qualitative portion of the study is supported by the following questions. The subsequent frames will provide the views and opinions of the respondents. Themes or categories emerged from these responses. Authors were also cited in concordance with the given probes. 1.Why is TAKS given as a statewide test? 2.What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?) 3.What problems have occurred related to or because of TAKS? 4.What changes were caused by this statewide testing? 5.What are your recommendations to improve this statewide testing? 6.What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test?
  • The first question asked was: Why is TAKS given as a statewide test? Responses given include the following: TAKS is given as a tool to gauge knowledge in the core areas. TAKS is considered as a means to determine the school’s status (Exemplary, etc.). TAKS is a means to assess the state curriculum or standards. This statewide test is mandated by law and is aligned with NCLB. Abrams and Badaus (2003) and Madaus and O’Dwyer (1999) cited reasons (in italics) why tests are given.
  • The second question asked was: 2. What are the intended consequences of this statewide testing? Respondents had these views: statewide testing is intended to eventually result in ELLs performing as well as the rest of the students; ELLs can improve academically and eventually join the mainstream; and TAKS is a requirement for graduation. Abedi (2003) and Harlow and Jones (2003) gave reasons why for the reasons why high-stakes tests are given.
  • 3. What problems have occurred related to or because of TAKS? Respondents identified the following as problems encountered by ELLs due to TAKS: (1) higher dropout rate; (2) decrease in graduation rate for ELLs; and, (3) lower self-esteem of ELLs
  • Rothstein (2002), Jacob (2001) and Anderson (2004) mentioned problems related to high-stakes testing: drop-out rate.
  • The fourth concern or issue posed was: What changes were caused by this statewide testing? The predominant responses were as follows: Schools experienced the negative reality that: There is a high failure rate among ELLs. Lower self-esteem and exasperation on the part of ELLs. High pressure on the school and ELLs. Too much emphasis is placed on test performance.
  • Anderson (2004), Lane and Stone (2002) and Flores and Clark (2003) affirmed the answers of the respondents.
  • When asked the question: What are your recommendations to improve this statewide testing? The respondents gave two major recommendations: (1) deferment of the test, possibly a different but fair test; and, (2) better assistance from the school through the teachers and the curriculum, modifications in teaching and possibly a paced curriculum for ELLs.
  • Anderson (2004) gave the positive and negative consequences about high stakes testing in his own study about ELLs.
  • A final question asked in the qualitative portion of the study was: What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test? The recommendations seen as more helpful were: Specific interventions for ELLs; Quality instruction; A more intensive English program; and, Help from home and meaningful tutoring in school. Flores and Clark (2003) and Popham (2003) cited proper focus on ELLs regarding statewide testing in order to undertake specific action plans to help ELLs.
  • Implications or what should be considered in dealing with ELLs are: 1.Performance of schools in high stakes testing is affected by size and proportion of ELLs taking the test. 2.Consider extended deferment of standardized tests administered in English to ELLs. 3.Learning is transmitted through communication. 4.More active role of LPAC in monitoring ELLs in schools. 5.Specific action plans to improve situation of ELLs.
  • Implications or what should be considered in dealing with ELLs are: 1.Performance of schools in high stakes testing is affected by size and proportion of ELLs taking the test. 2.Consider extended deferment of standardized tests administered in English to ELLs. 3.Learning is transmitted through communication. 4.More active role of LPAC in monitoring ELLs in schools. 5.Specific action plans to improve situation of ELLs.
  • Based on the results of the study, the researcher recommends further study on the following: 1.Additional supports to ensure ELLs to pass high stakes testing. 2.Identify what data are needed to make fair decisions about ELLs. 3.Determine reasons why ELLs scored lowest among student groups in TAKS. 4.Explore different approaches in campuses in dealing with ELLs in terms of curriculum and instruction.
  • Based on the results of the study, the researcher recommends further study on the following: 1.Additional supports to ensure ELLs to pass high stakes testing. 2.Identify what data are needed to make fair decisions about ELLs. 3.Determine reasons why ELLs scored lowest among student groups in TAKS. 4.Explore different approaches in campuses in dealing with ELLs in terms of curriculum and instruction.
  • Additional recommendations for further study include the following: 5.Compare performance of ELLs vs. non-ELLs based on different objectives of TAKS. 6.Impact of high stakes testing as viewed by parents and students. 7.Explore different instruments to measure performance of ELLs. 8.Determine if there is significant difference in performance of different student groups in statewide testing.
  • Additional recommendations for further study include the following: 5.Compare performance of ELLs vs. non-ELLs based on different objectives of TAKS. 6.Impact of high stakes testing as viewed by parents and students. 7.Explore different instruments to measure performance of ELLs. 8.Determine if there is significant difference in performance of different student groups in statewide testing.
  • This study affirmed the expected outcome that a significant relationship existed between the percentage of ELLs enrolled in a school and the percentage of all students passing the 10th grade TAKS tests in both core areas of English Language Arts and Mathematics. The regression analysis predicted that as the percentage of ELLs in a school increased, the performance in the statewide, high-stakes testing in terms of all students passing the 10th grade TAKS tests decreased. The respondents of the study considered the Texas Assessment of Knowledge and Skills (TAKS) as a tool to gauge knowledge in the different core areas. English language learners were expected to have at least average scores on TAKS. There was a difference in the expected and actual results; respondents observed dismal or failing performance of ELLS in the actual results in TAKS. This was evident by the high failure rate of ELLs in their respective schools. Higher dropout rate and lower graduation rate of ELLs were problems encountered due to TAKS. Respondents favored a different test for ELLs, possibly given at a later date after ELLs had studied in the country for at least several years. Respondents believed that interventions were needed to help ELLs perform better. Both the school and the home, together with the community have to be involved in preparing ELLs to be better prepared for their present and future roles in the American society.
  • Transcript

    • 1. A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas A Dissertation Defense by Arthur L. Petterway William Allan Kritsonis, PhD Dissertation Committee Member May 04, 2007
    • 2. Committee Members
      • M. Paul Mehta, Ph.D.
      • (Dissertation Chair)
      • William Allan Kritsonis, Ph.D.
      • (Member)
      • Douglas Hermond, Ph.D.
      • (Member)
      • David Herrington, Ph.D.
      • (Member)
      • Camille Gibson, Ph.D.
      • (Outside Member )
      May 04, 2007
    • 3. Dissertation Defense Format
      • Theoretical Framework
      • Purpose of the Study
      • Quantitative Research Question
      • Null Hypotheses
      • Methods: Subjects
      • Methods: Instrumentation
      • Methods: Quantitative
      • VIII. Quantitative Findings
      • IX. Qualitative Research Question
      • X. Qualitative Pilot Study
      • XI. Qualitative Findings/Review of Literature
      • XII. Practical Recommendations
      • XIII. Recommendations for Further Study
    • 4. Theoretical Framework A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas Q U A N T I T A T I V E Q U A L I T A T I V E % ELLs Enrolled In a High School Views/ Opinions Administrators Teachers District Personnel % All 10th Grade Students Passing TAKS ELA/ Math Impact of Statewide Testing On ELLs Student Performance Explanatory Design May 04, 2007
    • 5. Purpose of the Study
      • The purpose of this study was to determine the impact of high-stakes testing on English Language Learners.
      May 04, 2007
    • 6. Quantitative Research Question
      • Is there a relationship between the percentage of English Language Learners enrolled in a school and the percentage of all students passing the 10 th grade TAKS test in the core areas of English Language Arts and Mathematics given in 2003, 2004, 2005, and 2006?
      May 04, 2007
    • 7. Null Hypotheses
      • H 01 : There is no statistically significant
      • relationship between the percentage
      • of English Language Learners
      • enrolled in a school and the
      • percentage of all students passing
      • the 10 th grade TAKS test in English
      • Language Arts given in 2003, 2004,
      • 2005, and 2006.
      May 04, 2007
    • 8. Null Hypotheses
      • H 02 : There is no statistically significant
      • relationship between the percentage
      • of English Language Learners
      • enrolled in a school and the
      • percentage of all students passing
      • the 10 th grade TAKS test in
      • Mathematics given in 2003, 2004,
      • 2005, and 2006.
      May 04, 2007
    • 9. Method
      • Subjects of the Study
      • Quantitative – 173 Urban High Schools in Texas
      • Qualitative (N = 55)
        • 6 principals
        • 9 assistant principals
        • 6 ESL district personnel
        • 15 ESL certified teachers
        • 19 Non-ESL certified teachers
      May 04, 2007
    • 10. Method
      • Instrumentation
      • Quantitative data were accessed and retrieved from the TEA website regarding the major urban high schools in Texas.
      • Qualitative data were derived from the on-line open-ended questionnaire, focus group, and individual interviews.
      May 04, 2007
    • 11. Method Quantitative
      • Descriptive Statistics
      • Correlation Statistics
        • Pearson r Coefficient
      • Simple Regression Analysis
      • ŷ=a+b 1 x 1
      May 04, 2007
    • 12. Method Quantitative
      • Predictor Variable – The percentage of English Language Learners enrolled in school.
      • Outcome Variable – The percentage of all students passing the 10 th grade TAKS test in English Language Arts and Mathematics.
      May 04, 2007
    • 13. Major Findings: Descriptive Statistics English Language Arts 78.05 6.94 2006 59.39 7.62 2005 68.28 8.29 2004 62.87 8.30 2003 Percent of All Students Passing 10 th Grade TAKS Percent of ELLs Enrolled in Schools Year
    • 14. Major Findings: Descriptive Statistics Mathematics 50.13 6.94 2006 47.68 7.62 2005 53.57 8.29 2004 61.85 8.30 2003 Percent of All Students Passing 10 th Grade TAKS Percent of ELLs Enrolled in Schools Year
    • 15. Major Findings: Correlation Pearson r Coefficients
      • Year ELA Mathematics
      • 2003 -0.349** -0.293**
      • 2004 -0.392** -0.351**
      • 2005 -0.297** -0.382**
      • 2006 -0.398** -0.356**
      • ** Correlation is significant at the 0.05 level
      May 04, 2007
    • 16. Qualitative Research Question
      • What are the anticipated and observed consequences of the statewide testing specifically, TAKS, on ELLs, ESL curriculum, and instruction as viewed by certified ESL teachers, non-certified ESL teachers who teach ELLs, school administrators, and district ESL personnel?
      May 04, 2007
    • 17. Pilot Study
      • Qualitative
      • Developed an open-ended online questionnaire
      • Two schools-Expert opinions:
      • 2 principals
      • 2 assistant principals
      • 1 district ESL personnel
      • 4 ESL certified teachers
      • 4 Non-ESL certified teachers
      • Trustworthiness
      May 04, 2007
    • 18. The major question was explored using the following six probes:
      • Why is TAKS given as a statewide test?
      • What are the intended consequences of this statewide testing? (Or what has happened because of TAKS?)
      • What problems have occurred related to or because of TAKS?
      • What changes were caused by this statewide testing?
      • What are your recommendations to improve this statewide testing?
      • What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
      May 04, 2007
    • 19. Major Findings-Probe: 1. Why is TAKS given as a statewide test?
      • TAKS is given as a tool to gauge knowledge in the core areas.
      • TAKS is considered as a means to determine the school’s status (Exemplary, etc.).
      • TAKS is a means to assess the state curriculum or standards.
      • This statewide test is mandated by law and is aligned with NCLB.
      May 04, 2007
    • 20. Major Findings-Review of Literature: 1. Why is TAKS given as a statewide test?
      • Abrams & Madaus (2003)-Today’s widespread implementation of standards-based reform and the federal government’s commitment to test-based accountability ensure that testing will remain a central issue in education for the foreseeable future. Test results can provide useful information about student progress toward meeting curricular standards.
      May 04, 2007
    • 21. Major Findings-Review of Literature: 1. Why is TAKS given as a statewide test?
      • Madaus & O’Dwyer (1999)- Current emphasis on testing as a tool of education reform continues a long tradition of using tests to change pedagogical priorities and practices.
      May 04, 2007
    • 22. Major Findings-Probe: 2. What are the intended consequences of this statewide testing?
      • Statewide testing is intended to eventually result in ELLs performing as well as the rest of the students.
      • ELLs can improve academically and eventually join the mainstream.
      • TAKS is a requirement for graduation.
      May 04, 2007
    • 23. Major Findings-Review of Literature: 2. What are the intended consequences of this statewide testing?
      • Abedi (2003)- In a positive light, valid assessment may provide diagnostic information that can inform instruction and classification.
      May 04, 2007
    • 24. Major Findings-Review of Literature : 2. What are the intended consequences of this statewide testing?
      • Harlow & Jones (2003)- We also need to recognize that when test scores are linked to high-stakes consequences, they can weaken the learning experiences of students, transform teaching into test preparation, and taint the test itself so that it no longer measures what it was intended to measure.
      May 04, 2007
    • 25. Major Findings-Probe: 3. What problems have occurred related to or because of TAKS?
      • Higher dropout rate
      • Decrease in graduation rate for ELLs
      • Lower self-esteem of ELLs.
      • Dismal or failing performance of ELLs
      May 04, 2007
    • 26. Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?
      • Anderson (2004)- For English Language Learners, the additional requirements of an exit examination could increase dropout rates.
      • Rothstein (2002)- Dropout rates are climbing throughout the United States and many researchers hold high-stakes testing at least partly to blame.
      May 04, 2007
    • 27. Major Findings-Review of Literature: 3. What problems have occurred related to or because of TAKS?
      • Jacob (2001)- Some researchers found that dropout rates were 4 to 6 percent higher in schools with high school graduation examinations. Another study reported that students in the bottom quintile in states with high-stakes testing were 25% more likely to drop out of high school than were their peers in states without high-stakes testing.
      May 04, 2007
    • 28. Major Findings-Probe : 4. What changes were caused by this statewide testing?
      • Schools experienced the negative reality that there is a high failure rate among ELLs.
      • Low self-esteem because of low scores and tests caused frustration and exasperation on the part of ELLs.
      • Pressure on the school and ELLs.
      • Emphasis is placed on test performance.
      • ELLs need more time to learn English
      May 04, 2007
    • 29. Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?
      • Anderson (2004)- Labeling schools can have an impact on teacher and student morale.
      • Flores & Clark (2003)- Teachers have also reported that the high-stakes nature of some assessments can have a negative impact on student morale.
      May 04, 2007
    • 30. Major Findings-Review of Literature: 4. What changes were caused by this statewide testing?
      • Hood (2003)- Although some teachers have reported that their English language learners can reach the high standards set for them, they may need more time than other students.
      • Lane & Stone (2002)- Certainly, poor test scores or poorly explained assessment systems can result in decreased student motivation.
      May 04, 2007
    • 31. Major Findings-Probe : 5. What are your recommendations to improve this statewide testing?
      • Deferment of the test, possibly a different but fair test.
      • Better assistance from the school to provide for teacher collaboration.
      • Modifications in teaching strategies.
      • Improved and paced curriculum for ELLs.
      May 04, 2007
    • 32. Major Findings-Review of Literature: 5. What are your recommendations to improve this statewide testing?
      • Anderson (2004)- Positive consequences that were identified included more teacher collaboration, changes in curriculum and instruction, better alignment between ESL and content area curricula and more focus on reading and writing. Negative consequences included student and teacher frustration, more teaching to the test occurring, and a narrowed curriculum.
      May 04, 2007
    • 33. Major Findings-Probe: 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
      • Specific interventions for ELLs.
      • Quality instruction.
      • Develop a more intensive English program.
      • Support from home.
      • Provide a more meaningful tutoring program in school.
      May 04, 2007
    • 34. Major Findings-Review of Literature: 6. What needs to be done for the ELLs to improve their performance in general and specifically for this statewide test?
      • Flores and Clark (2003) found that teachers were not against accountability and viewed it as distinct from statewide testing, but also thought that an over-emphasis on testing resulted in unbalanced curriculum and inappropriate instructional decisions.
      May 04, 2007
    • 35. Major Findings-Review of Literature: 6. What needs to be done for the ESL students to improve their performance in general and specifically for this statewide test?
      • Popham (2003)- In order for teachers to make specific changes to instruction, the assessments needs to be clear as to what skills are being assessed.
      May 04, 2007
    • 36. Practical Recommendations
      • School leaders should consider that the performance of schools on high stakes testing is affected by size and the proportion of ELLs taking the test.
      • School leaders should consider extended deferment of standardized tests administered in English to ELLs.
      May 04, 2007
    • 37. Practical Recommendations
      • School leaders should consider taking a more active role with LPAC in the monitoring of ELLs in schools.
      • School leaders should implement specific action plans to improve situation of ELLs.
      May 04, 2007
    • 38. Recommendations for Further Study
      • A study could be conducted to explore additional supports needed to ensure ELLs to pass high stakes testing.
      • A study could be conducted to identify what data are needed to make fair decisions about ELLs.
      May 04, 2007
    • 39. Recommendations for Further Study
      • A study could be conducted to determine reasons why ELLs scored lowest among student groups on TAKS.
      • A study could be conducted to Explore different approaches on campuses in dealing with ELLs in terms of curriculum and instruction.
      May 04, 2007
    • 40. Recommendations for Further Study
      • A study could be conducted to compare the performance of ELLs vs. non-ELLs based on different objectives of TAKS.
      • A study could be conducted to understand the impact of high stakes testing as viewed by parents and students.
      May 04, 2007
    • 41. Recommendations for Further Study
      • A study could be conducted to Explore different instruments to measure performance of ELLs.
      • A study could be conducted to determine whether the provision of English Language instruction to parents of ELLS would have a significant effect on student achievement on high stakes testing.
      May 04, 2007
    • 42. A Mixed Methods Analysis of the Impact of High Stakes Testing on English Language Learners in Major Urban High Schools in Texas Q U A N T I T A T I V E Q U A L I T A T I V E % ELLs Enrolled In a High School Views/ Opinions Administrators Teachers District Personnel % All 10th Grade Students Passing TAKS ELA/ Math Impact of Statewide Testing On ELLs Student Performance Explanatory Design May 04, 2007