Does the NCLB toolkit work? Is there a better way?
•Download as PPT, PDF•
0 likes•343 views
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Does the NCLB toolkit work? Is there a better way?
Similar to Does the NCLB toolkit work? Is there a better way? (20)
More from Marshall S. (Mike) Smith
Recently uploaded
Recently uploaded (20)
Does the NCLB toolkit work? Is there a better way?
- 1. Should We Focus on theShould We Focus on the NCLB “Toolkit” or onNCLB “Toolkit” or on Improving Learning?Improving Learning? Marshall S. Smith November 30, 2006 American Enterprise Institute
- 2. Marshall Smith The William and Flora 2 • Excellent, thoughtful set of papers • Conclusions – almost universal!!!! – Implementation weak – Frequency of use – small to miniscule – No discernable achievement effect
- 3. Marshall Smith The William and Flora 3 What Did We Expect?What Did We Expect? • Complicated, patchwork, non-systemic interventions
- 4. Marshall Smith The William and Flora 4 MoreoverMoreover —— No Support from Prior ResearchNo Support from Prior Research
- 5. Marshall Smith The William and Flora 5 Time to Stop Messing AroundTime to Stop Messing Around on the Marginson the Margins • What do we know about achievement over past decades? • What do we know about how to improve learning?
- 6. Marshall Smith The William and Flora 6 Recent Achievement TrendsRecent Achievement Trends Look at NAEP subgroups for accurate picture. Approximately 1994-2005. Both NAEPs. Math gains large – 1 to 2+ grade levels. Reading modest but significant. 0.7-1+ gl. 4th grade gains considerably larger than 8th grade.
- 7. Marshall Smith The William and Flora 7 Reading Main NAEP Reading Longitudinal Math Main NAEP Math Longitudinal 2005-1994 gains 2004-1994 gains 2005-1992 gains 2004-1994 gains National 5 8 18 10 White 5 9 19 10 African American 15 15 27 12 Hispanic American 15 19 24 20 NAEP Gains for Grade 4 (Main) and Age 9 (Longitudinal) – Reading and Mathematics (10-11 points = 1 grade level)
- 8. Marshall Smith The William and Flora 8 Achievement Before and AfterAchievement Before and After Standards-Based ReformsStandards-Based Reforms (SBR)(SBR) Compare time periods 1978-1994 with 1994-2005 — Longitudinal NAEP Math — Gain slopes before and after SBR similar — slight edge to after SBR Reading — Gain slopes considerably larger after SBR SBR based on these data SBR is a qualified success
- 9. Marshall Smith The William and Flora 9 Achievement Before and AfterAchievement Before and After NCLBNCLB • Compare achievement slopes 1994- 2002 with 2002-2005 • Math slopes approximately the same • Reading slopes greater before NCLB • Jury out (need 2007 data) but certainly no evidence of success!
- 10. Marshall Smith The William and Flora 10 Main NAEPMain NAEP Reading Scale Scores – Grade 4Reading Scale Scores – Grade 4 Groups Years Gains 1994 2002 2005 2002- 1994 2005- 2002 Nation 214 219 219 +5 0 White 224 229 229 +5 0 African American 185 199 200 +14 +1 Hispanic American 188 201 203 +13 +2
- 11. Marshall Smith The William and Flora 11 Main NAEPMain NAEP Reading Scale Scores – Grade 8Reading Scale Scores – Grade 8 Years Gains Groups 1994 2002 2005 2002- 1994 2005- 2002 Nation 260 264 262 +4 -2 White 267 272 271 +5 -1 African American 236 245 243 +9 -2 Hispanic American 243 247 246 +4 -1
- 12. Marshall Smith The William and Flora 12 Summing Up So FarSumming Up So Far • Math and reading gains larger than many believe. Substantial gains in 4th grade in reading and math during SBR years, less but still important gains in 8th grade. • Gaps closing slowly over past 12 years in 15 of 16 comparisons in NAEP. Between 10% and 30% from 1994-2005. Not fast enough. • SBR a qualified success. Three of four cases gain slopes are greater after the SBR reforms started than before. • No evidence of NCLB effectiveness. Substantial decline in slope of gains since NCLB passed for reading in both grades – no difference for math.
- 13. Marshall Smith The William and Flora 13 One OptionOne Option Stay the Course – with minimal correctionStay the Course – with minimal correction • Standards-based reform works. • It is too early to see results of NCLB (“toolkit”, tough accountability and Reading First). We need more time but we know they work. • Therefore up the ante – tell locals to implement better -- make sanctions tougher on “misuse” of NCLB “tools” -- increase intensity of accountability to make sure results come faster. • Create and require national tests because, I guess, they believe, they will make everyone work harder and learn more. Some folks argue:
- 14. Marshall Smith The William and Flora 14 What Should We Do?What Should We Do? • Don’t stay the course! Take bold steps but build on strength. • Most educational policy is on the margin – little programs, little add-ons, create chaos and have no effects. Throw them out! • System wants us to continue playing on the margins – this satisfies adults, special interests, and other inside beltway inhabitants.
- 15. Marshall Smith The William and Flora 15 Think About Improving theThink About Improving the System to Improve LearningSystem to Improve Learning • Learning = f(S, C, M, T) – System – Content – Motivation – Time • Shoot for big effects
- 16. Marshall Smith The William and Flora 16 SystemSystem • Use positive incentives throughout the system – reward rather than punish – encourage creativity and initiative rather than regulate rigidity. • Support information-based continuous improvement at all levels. • Retain SBR teacher qualification requirements, dis- aggregation, school, district and state reports, increase transparency. • Support longitudinal student and teacher-based data systems. • Hold accountable those who make decisions – not only those who carry them out. • Remove add-ons! Remember – the point is to help schools improve and children learn!
- 17. Marshall Smith The William and Flora 17 ContentContent • Focus on Language, Language, Language!!! • Start early and continue throughout – preschool and beyond with good role models, lots of oral language, comprehension, complex sentence structure, lots and lots of reading, academic language. • E.D. Hirsch, Catherine Snow, etc.
- 18. Marshall Smith The William and Flora 18 Motivation and Desire to LearnMotivation and Desire to Learn • Ensure healthy children – teeth, eyes, ears, childhood disease. • Increase Music, Art, Learning by Doing, enhance creativity and learning how to learn. Analysis, synthesis. Give students more control over their learning.
- 19. Marshall Smith The William and Flora 19 TIMETIME • Create incentives to extend time of the regular school day and year (learn from KIPP and others) • Accelerate learning for those who are bored — using technology creatively
- 20. Marshall Smith The William and Flora 20 CLIMB Out of the BOXCLIMB Out of the BOX Don’t settle for small effects!
Editor's Notes
- What did we expect? Difficult, patchwork interventions involving. Many parties, private and public Extra demands on time and skills of parents Creation of new organizational entities and relationships Little necessary commitment of energy by student. State implementation impossible after state departments denuded. In some interventions reduction of obligation of mainstream teachers.
- One might argue that with time and more effort we can overcome these challenges but the extensive research evidence from the past suggests otherwise. Unless such interventions are tightly connected to and heavily reinforced and reinforcing of the regular school program, the literature suggests no effects on achievement. After school programs, summer programs, pull out tutoring, all nil effects unless tightly and thoughtfully connected to regular schooling. . Magnet schools, charters, vouchers all on average have no effect. Breaking up schools, creating charters, reconstitution, state interventions to date all have no or tiny effects.
- Implications for Reauthorization of Title I “Tool kit” focus is too confining. The three “tools” (perhaps better called strategies), are themselves piecemeal and would not be expected to have significant influence on underserved students in failing schools. Moreover, the focus on the “tool kit” takes as a given the rest of NCLB, including the continued focus on standards based reforms, the emphasis in Reading First on early reading, the use of only punitive incentives in the accountability system, as well as a host of other issues. As a beginning it I believe that we should step back, review our basic assumptions, look at the evidence gathered from these papers and other sources over the past few years, focus on improving the conditions for teaching and learning, eliminate interventions that have not worked and propose changes that have promise of increasing achievement. In other words we should look at the evidence and make changes that seem to be dictated by the evidence. I don’t pretend that I do this job here but let’s start.
- Use two NAEP measures – Main (4th and 8th grade), and Longitudinal (9 and 13 years old) Baseline for start of SB reforms – 1994 for reading and 1992 for math SBR effects pre-NCLB – 1994-2002 for reading, 1992-2003 for math Post NCLB – 2002-2005 for reading, 2003-2005 for math: use 2005 as end
- <number> Have NAEP scores increased since the beginning of SBR? Start with fourth grade. Table shows data from Main and Longitudinal NAEP. 10-11 points equals approximately one grade level!! Note that gains for sub-groups generally greater than for national scores. This is due to increased percentages of Hispanics over the past 10-12 years in US schools. If we want to measure the effects of schools we should always look at the sub-group scores. If we want to measure changes in national human capital (as assessed by test scores) then we should look at the national numbers. 4th grade -- clear substantial gains for math and reading on both assessments Gains larger for African and Hispanic Americans than whites. African American and Hispanic American gains in reading roughly 1.5 grade levels on both tests over the same time period. On main NAEP, math gains for both groups greater than 2 grade levels and this is a more challenging test On math longitudinal African American over a grade level and Hispanic American almost 2 grade levels Some gap closing indicated by larger gains of minority groups in all eight comparisons. ** Question for audience: Why are the gains of all of the subgroups larger or exactly as large as the gains for the national sample? Reason the distributions of people of different groups changed. Main and Longitudinal NAEP gains from 1992(94) – 2004(5) substantial: Reading: Hispanic and African Americans: 4th grade: 1+ grade levels. 8th grade 0.7 grade levels. Math: Hispanic and African Americans: 4th grade: 2+ grade levels. 8th grade: 1+ grade levels. Gap: Closing by roughly 15-30%: Whites gain less than “minorities” Gap closing greater in 4th grade than 8th grade. Greater in math than reading. An alternative way of looking at gap closing. Fourth grade Hispanic and African Americans in 2005 score the same as 4th grade whites in 1992.
- Using longitudinal NAEP we can compare rates of gain before and after SBR (1992/1994). Absent other explanations, based on these results, we can consider SBR a qualified success. For all three sub-groups in both grades in reading the rate of gain of achievement was clearly greater from 1994-2005 than from 1978-1994. For math the rates of gains are xxxxxx fill in –
- Using Main NAEP we can contrast the rates of gain of 1994-2002 with 1992 -2005 for reading and 1993-2003 with 2003-2005 for math. This gives us estimates of the effects of NCLB on student achievement. If the rates of gain are equal for the two time periods there is no effect. If the rate of gain is greater after NCLB was passed then there is a positive effect -- if the rate of gain after NCLB is less than before then there is a negative effect. For mathematics for both 4th and 8th grades the rates of gain are about equal before and after NCLB indicating no effect. For reading for both 4th and 8th grades the rates of gain are less after NCLB indicating a negative effect.
- <number> What about reading? Note that the testing dates change -- these are closer dates to the real initiation of the two different reforms. Note gains from 1994-2002 have considerably steeper slope than gains from 2002-2005 – for all sub-groups this suggests that after NCLB was passed the rates of growth for 4th grade reading slowed substantially. Note also gap closing -- 10 point gains in gap closing for both African and Hispanic Amerians. Roughly 25 to 28% closing. Equivelent to almost a grade level. Difference from 39 to 29 points for AA -- for HA 36 to 26 points.
- <number> The same pattern of reduction in the slopes appears for reading in 8th grade. In fact, in this case the slopes for the gains has turned negative for the NCLB years. The data suggest that the gains of the 1990s (such as they are) halted and reversed after NCLB. Gap closing much less than in 4st grade for African Americans -- increases by a point for Hispanic Americans.
- <number>
- <number> The premises are plausible. The conclusions are fantasy at best, ideological at worst.
- System -- structure, alignment, use of incentives, Content – delivery of content – teacher quality, knowledge, commitment, other forms of delivery, curriculum, Motivation -- of student, desire to work and learn, sense of control, quality of health affecting learning Time -- length of time to learn content – varies among students, depending largely on prior knowledge, fixed in US. Big effects -- education complex – small effect sizes disappear over time and when we try to move them from place to place. Following ideas based on some evidence and on logic and have promise for showing big effects.
- Positive incentives -- need to think this way rather than punitively. How about incentives for schools and districts for raising graduation rates (in places that have reasonably challenging exit exams?) How about rewarding schools that are improving rapidly? Use formative assessments. Overall need to support strengthening the quality of the systems -- data infrastructures, capacity to understand information and to take steps to improve based on it -- continuous improvement cycles (Toyota assembly lines – invert management so users suggest changes.) True formative assessment -- these things cannot be mandated but they can be demonstrated and rewarded. Creat demonstrations -- America 2000 – 100 extraordinary high schools – but don’t regulate them, use incentives. Invert system -- who should be accountable for mandating programs and policies that do not work?
- Early childhood studies -- low income children one half the vocabulary of middle income – Vocabulary Oral speaking – good role models. Time for science, history, E.D. Hirsch Scores plummeting in middle school grades -- in math as well as english -- LA district ELL kids – Uri Triesman example.
- Growing evidence about health problems as impediments for learning. Early screening repeated evey two or three years with referral care. Teeth alone as a critical problem. Easy to train teachers to spot symptions. Would require incentiives for public health systems to work with school system. Music, Art -- critical to motivation. Ditto on learning by doing -- talking, problem solving real problems, doing what scientists do, learning at your own pace -- Why not have the feds create some really interesting assessments that measured creativity and the capacity to