Making the Case for Quality
The	District’s	Homework:	Studying	Levels	of	Student	Engagement

 One	of	Westfield	Washington’s	core	beliefs	centers	on	pro...
measurement once they began incorporating the Baldrige criteria.                             One example of moving from la...
Figure 1—Balanced Scorecard (continued)
Figure 2—Demographic information for                                  Figure 3—Mean NWEA RIT (Rasch UnIT) growth
Upcoming SlideShare
Loading in …5

Growing Indiana School District Stretches For Improvement Using Baldrige Model


Published on

Growing Indiana School District Stretches For Improvement Using Baldrige Model

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Growing Indiana School District Stretches For Improvement Using Baldrige Model

  1. 1. Making the Case for Quality October 2007 Growing Indiana School District Stretches for Improvement Using Baldrige Model by Janet Jacobsen The Westfield Washington School District (WWSD) was in an enviable position: Located in a At a Glance . . . booming, affluent suburb of Indianapolis, Indiana, the district watched student achievement scores rise almost as quickly as the bricks for one new school building after another. While some might view this as a time to put improvement efforts on autopilot, superintendent Dr. Mark Keen instead recognized • With test scores rising and new schools opening, the a perfect opportunity to stretch even further by adopting a continuous improvement model with even Westfield Washington higher levels of quality and accountability, the Baldrige Criteria for Performance Excellence. School District decided it was the perfect time to About Westfield Washington School District take on greater challenges and reach for higher levels of continuous quality In the past 11 years WWSD has experienced phenomenal growth—from 2,400 students in four schools improvement by embracing in 1996 to more than 5,600 students and eight buildings at the start of the 2007-08 school year. Keen, the Baldrige Criteria for who was honored as Indiana’s Superintendent of the Year in 2006, has led the district during this time Performance Excellence. of rapid expansion. Under his direction the district embraced the principles of systematic change found • After adopting the Baldrige model, district in the Schlechty Model from the Center for Leadership in School Reform (CLSR). From here, the leaders learned they district accepted the challenge of the Baldrige criteria, submitting its first application for the Malcolm were relying too heavily Baldrige National Quality Award in the spring of 2007. on lagging measures of student performance rather Selecting Baldrige as the District’s Model than on more timely, in-process measures. • Results from benchmarking Though WWSD flourished under CLSR principles, it was prepared to take on other challenges. “In showed Westfield education, you’ve never arrived,” says Todd Lambert, the district’s director of systems integration Washington’s fifth and and a certified Baldrige examiner. He recalls that Baldrige was the best choice among several quality sixth graders demonstrated higher gains between models. “We selected it not because it was the one that fit us the best, but because Baldrige was the spring and fall testing one we felt would stretch us the most,” Lambert explains. He notes that the philosophy behind the than did their counterparts district’s Baldrige efforts is simple: Stay focused and improve on what the district already does well. in comparable districts. District officials attribute these results to a focus on Gaining stakeholder buy-in from community members and teachers has been and remains an ongoing high-quality lessons and journey. Lambert, a former principal in the district, stresses the importance of building trust with student engagement. parents and the community by demonstrating that the district puts students first. He says the teaching staff is highly supportive due, in part, to good communication from principals and other administrators: “The key is that we don’t just develop new processes or tools for teachers; we give them information on why a change is necessary and why we are doing something in a certain way.” The American Society for Quality Page 1 of 5
  2. 2. The District’s Homework: Studying Levels of Student Engagement One of Westfield Washington’s core beliefs centers on providing students • Organization of knowledge with engaging work. According to the Center for Leadership in School • Content and substance Reform (CLSR), students are more engaged in the classroom when • Affiliation lessons are prepared and presented using certain design qualities. The residual effects include higher achievement and improved discipline. • Clear and compelling standards To test this premise, the district has conducted four engagement studies in Lambert reports that the teachers quickly processed the ratings and recent years. During the 2006-07 school year, the district’s intermediate began using the highly-rated qualities more in the classroom to improve school (grades 5 and 6) embarked on a study to learn which design levels of engagement: “It was an incredibly strong study; we learned a qualities correlated with the highest levels of student engagement. Todd great deal about our students.” Lambert, then the school’s principal, says the idea was to have students rate their levels of engagement; at the same time, the teacher would According to Lambert, the district’s engagement studies tie in nicely with track which design qualities he or she was using during that particular Baldrige because the model is not prescriptive. He describes Baldrige lesson. as simply a model for how to develop and deploy processes, learn from them, and make adjustments. The studies also correlate with the idea of The study utilized the technology of Promethean boards, which are using data that are constantly available, rather than lagging data. In this “smart white boards” with hand-held remote response units for each case, the district can capture student engagement data on a daily basis student. At the conclusion of a lesson, the teacher would ask students to if needed, updating them as often as desired. “Of course there is a leap rate their engagement at one of three levels: of faith, as we believe that if students are engaged they will perform at a A—engaged most of the time higher level and have a higher level of satisfaction,” notes Lambert. B—engaged some of the time Average Number of Students Engaged C—not at all engaged 100% 92.5% As students entered their ratings, the teacher tracked the design quality/ 88.1% qualities used during that particular lesson with a spreadsheet on a 80% 76.1% Percentage Engaged in Category desktop computer. Once the student ratings were complete, the teacher 73.2% 72.3% 71.8% 71.5% created a graph depicting what students thought of the just-completed 63.4% 62.8% 61% lesson, thus providing a valuable opportunity for the instructor and (Most of the Time) 60% students to discuss the ratings. The graph at right shows the results from the design quality study at Westfield Intermediate School. The graph depicts the percentage of 40% students who were engaged most of the time when a particular strategy was the primary design quality during the lesson. Out of the 10 CLSR 20% design qualities, listed below, “choice” produced the highest levels of engagement, which Lambert says was not a surprise. CLSR design qualities: 0% • Choice ce ity y es us ge e n s ce rd iet nc tio nc oc oi an tic led da ta r ia Ch Va ue tF en rm bs ow fil an • Authenticity eq uc th rfo Af Su & St Kn Au od ns y Pe nd g elt Co Pr of llin of ta ov • Novelty and variety n rse pe io N en e nc m at ve nt Co ca iz Ad Co • Protection from adverse consequences an ifi r& gn om rg O ea Si Fr Cl • Affirmation of the significance of performance e n th tio of ec n ot • Product focus io Pr at m fir Af Westfield Washington’s Continuous Quality Journey tool to support district initiatives. Starting this school year, all school buildings feature data display boards, and the district created a balanced scorecard, shown in Figure 1, with goals and When WWSD embraced the Baldrige process, it re-established measurements for each goal listed. All school buildings also school plans and realigned its focus to help students reach their have a balanced scorecard for schoolwide initiatives. The goals individual growth targets. Now the district not only strives to are separated to keep principals focused on segmentation. “If meet individual needs but also focuses the teaching staff on you are going to individualize and customize delivery of instruc- differentiation and accountability with individual students. “It tion, you have to break your groups apart to see where there are [Baldrige] has really changed the core of everything, all the way strengths and weaknesses,” explains Lambert. to professional development, to our vision and mission state- Focusing on In-Process Measures ment, and just the overall way we do business,” states Lambert. In concert with the district’s overall plan for continuous quality While the district had routinely used data for decision making, improvement, each school uses the plan-do-study-act (PDSA) leaders realized they had a great deal more to learn about The American Society for Quality Page 2 of 5
  3. 3. measurement once they began incorporating the Baldrige criteria. One example of moving from lagging data to in-process measures Lambert says the district tended to focus too heavily on lagging involves writing assessments. In the past, writing measurements data. For example, WWSD students take statewide achievement were not standardized but were carried out classroom by classroom. tests in late September, and results typically arrive just before Now, WWSD routinely administers common writing assessments Thanksgiving. Before the Baldrige initiative, the data had never throughout the schools a minimum of six to eight times per year. really been used until January and could not have a timely The effects of this new practice are trickling down into staff impact on classroom instruction. District leaders now understand development activities as it has become apparent that the best way that measuring and monitoring must be routine and that lagging to achieve consistency in grading subjective tests, such as writing data will always be available. If they do a good job of measuring assessments, is through strong professional development. along the way, they can predict in a fairly accurate manner what Benchmarking to Demonstrate Achievement the lagging data will tell them. “We’re trying to get in the habit of measuring and monitoring on In addition to using PDSA charts and balanced scorecards, the a regular basis so that we can be agile. We used to think we were district’s quality improvement journey also features benchmarking agile, but in reality, we probably were not,” acknowledges Lambert. activities, one of which centered on a core belief of the district: providing students with engaging work. Through its affiliation Figure 1—Balanced Scorecard 2010 Targets (each has on-track Strategic Objectives Goals Measures targets for 2007, 2008, and 2009) Students pass ISTEP+ Fall ISTEP+ 93% passing in both LA and math 100% meet or exceed individual All students grow in math NWEA (3x year) RIT growth targets in math, Students master grade level and language arts language arts, and reading state and district standards in math and language arts WWS students demonstrate the 93% of students have a higher “value add” standard by performing Terra Nova 3rd, 6th, and 8th grades obtained score than anticipated better than expected if in the district score in target areas for more than one full year 93% of students score above 3 on Students perform well in applied skills Fall ISTEP+ the 0-6 point Applied Skills scale Students master grade Students demonstrate mastery Local assessments to be developed 93% meet or exceed WWS level state and district against 6 + 1 writing criteria and targets to be set writing standards standards in writing Students demonstrate skill Fall to Spring NWEA in Writing 100% meet or exceed individual RIT in writing process Process subgoal area growth target as applied to subgoal area Students demonstrate mastery of 93% of students meet or exceed Fall ISTEP+ IPI reading-related areas Student reading-related areas within LA mastery in Reading IPI Students exit grade level Learning and reading on level Students grow across all 100% of students meet or exceed Achievement Fall to Spring NWEA reading subgoal areas individual growth targets 100% of students meet or exceed All students learn and grow NWEA No significant differences individual growth targets in academic performance No identified student subgroup scores in for identified (SP) A maximum of 10% variance across math, language arts, reading vary more ISTEP+ NWEA subgroups of students aggregate student populations than 10% from any other subgroup Students experience Students successfully complete identified 100% of students experience post- success in post-high As recorded in transcripts post-high school level courses high school academic work school academic work Students are engaged in Action research in 2nd year for 100% of students report engagement Students are engaged in their work challenging, relevant work development of measures at WWS “best-in-class” levels 95% of students turn in Students demonstrate Teacher-piloted homework and homework on time Students demonstrate responsibility behaviors of responsible, behavioral measure ready for and discipline for learning 95% comply with school or hard-working student citizens implementation in Fall 07 campus rules and regulations Number of 5–12 students who participate in community service 93% of 5–12 students participate in projects (collection instrument to community service projects each year be developed by Spring 08) Students demonstrate behaviors of respectful, Students demonstrate the 95% of students receive + Citizenship grades in grades 1–4 compassionate, and behaviors of good citizenship citizenship grade honest student citizens Number of 5–12 students who report 95% of students in grades 5–12 positive school descriptors in student report school environments satisfaction surveys (collection instrument as positive places to be to be developed by Spring 08) figure continued on page 4 The American Society for Quality Page 3 of 5
  4. 4. Figure 1—Balanced Scorecard (continued) 2010 Targets (each has on-track Strategic Objectives/Goals Measures targets for 2007, 2008, and 2009) Annual Parent Survey 80% Satisfaction/Agreement Rate Parents are satisfied that expectations 3 Questions Survey at beginning of the year and requirements are met 80% Satisfaction/Agreement Rate compared to Annual Percent Survey Results Students are satisfied that expectations Annual District Student Survey (to be developed) 80% Satisfaction/Agreement Rate and requirements are met Constituent Satisfaction Annual Staff Member Survey 80% Satisfaction/Agreement Rate Staff members are satisfied that expectations and requirements are met Staff Focus Group Results 80% Satisfaction/Agreement Rate District survey Community is satisfied that expectations Annual community focus groups 80% Satisfaction/Agreement Rate and requirements are met with the Superintendent Faculty, staff, and administrators 100% of teachers must meet NCLB NCLB Definition are highly qualified highly qualified definition Teacher Retention Rates 90% teacher retention over 5 years Faculty, staff, and administrators work in a Staff Attendance Rates 98% attendance rates professional and supportive environment Employee Hiring Survey Results 80% Satisfaction/Agreement Rate and Development (Workforce) 90% satisfaction level with Participation Rates in Staff Development professional development Employees are satisfied that ongoing, Activities and Program sustained professional development 90% application of new skill level meets or exceeds expectations Staff Development Activities and Programs 100% of activities and programs meet or exceed NSDC Standards meet or exceed standards Key processes identify common and special “Customers” (students, parents, staff, cause variation by using appropriate All key processes are stable and capable Alignment of and community) of the school district control charts and histograms Support Systems are highly satisfied with optimized All system maps updated by process management systems System Maps are updated annually September 1 of each year Competitive salaries and benefits Support and Sustain Quality Educational Meets or exceeds local and state Fiscal Responsibility Instructional expenditures as % of budget Programming and Staff comparison benchmarks trends compared to benchmarks with CLSR, the district has taken the position that if lessons against two similar school districts using NWEA scores. The are prepared with certain design qualities, then students will be main difference between WWSD and the other districts? A focus more engaged in the classroom, the residual effects being better on engaging work. discipline, higher student achievement, and greater job satisfaction Figure 2 offers demographic information highlighting the for teachers. A basic premise of the CLSR model contends that similarities and differences among the three districts. Districts there are various levels of classroom engagement, ranging from one and two are somewhat comparable in terms of demographic meaningfully engaged all the way to rebellion. (See the sidebar numbers, although both are considered a bit more affluent and article, “The District’s Homework: Studying Levels of Student have smaller English as a Second Language (ESL) populations Engagement,” on WWSD’s recent engagement studies.) than WWSD. Figure 3 shows a fairly substantial achievement Twice each school year, the district administers a three-part gap between WWSD and two other districts on the amount of assessment from the Northwest Evaluation Association (NWEA). “growth” from fall to spring for fifth and sixth graders. Lambert says this is a powerful tool for the district, as it is a growth “We believe that our commitment to designing high-quality model assessment measuring student growth from one test to the lessons and focusing on student engagement has led to this gap,” next. NWEA publishes achievement data for students in three-digit Lambert states, adding that during the 2007-08 school year the scores with a separate score for each of the three categories: reading, district will collect these data in all grades. math, and language usage. Students in participating districts such as Washington Westfield are given the test in the fall to establish a Looking to the Future baseline and then again in the spring, with the growth rate between the two indicating the level of achievement. Not surprisingly, finding enough time to understand and Correlating Engaging Lesson Plans and Higher implement Baldrige principles has been one of the biggest Performance obstacles thus far in the Westfield Washington School District’s Baldrige journey. Lambert realizes that the district’s continuous quality improvement efforts also test the principals’ To test the premise that engaging work does indeed correlate to higher performance, WWSD conducted benchmarking studies communication abilities. “They are on the front lines The American Society for Quality Page 4 of 5
  5. 5. Figure 2—Demographic information for Figure 3—Mean NWEA RIT (Rasch UnIT) growth benchmarking with two other districts from September 2006 to May 2007 for that also use the NWEA test Westfield Washington’s fifth and sixth graders and two comparable districts Comparison Comparison Westfield District One District Two Washington School Math Reading Language School District District One 5th grade 3.3 1.9 3.1 Free/reduced lunch 17% 4% 14% District One 6th grade 3.8 1.2 1.7 Minority 8.9% 8.8% 12.7% District Two 5 grade 9.4 5.6 5.1 th Special education 18% 18.1% 17.4% District Two 6th grade 7.7 4.9 4.4 English as Second 1.9% 0.5% 4.2% Westfield Washington 5th grade 11.0 6.2 7.1 Language Westfield Washington 6th grade 10.5 6.1 5.5 communicating with parents and teachers while leading these The RIT Scale relates directly to the curriculum scale for each subject area endeavors. While not an obstacle, we’ve had to focus attention on and uses individual item difficulty values to estimate student achievement. issues such as communication and organization,” Lambert reports. It is an equal-interval scale, like the measurement of feet and inches: scores can be added together to calculate class or school averages. The district’s use of the Baldrige model is still relatively new, • Participate in the American Society for Quality’s Baldrige but it has already prompted school officials to closely examine in Education Network: habits of measuring and monitoring data. In the coming months, baldrige-education/index.html. WWSD will continue to move away from the use of lagging data • Contact Todd Lambert, director of systems integration at and continue a new focus on continuous growth and in-process Westfield Washington School District, for more information measures. Lambert predicts that when school improvement on the district’s continuous quality improvement journey: plans are prepared in the spring of 2008, they will not only reflect the Baldrige framework, but also demonstrate a deeper • To learn more about the Schlechty Center for School Reform, understanding by parents, students, and staff of what the district visit the organization’s Web site: is working to accomplish through continuous improvement. About the Author For More Information: Janet Jacobsen is a freelance writer specializing in quality and • Visit the Westfield Washington School District’s Web site at compliance topics. A graduate of Drake University, she resides in Cedar Rapids, Iowa. • To gain a wealth of free information on the Malcolm Baldrige National Quality Award, self-assessments, and the criteria for performance excellence, go to The American Society for Quality Page 5 of 5