Thesis

INFORMATION TO USERS
This manuscript has been reproduced from the microfilm master. UMI
films the text directly from the original or copy submitted. Thus, some
thesis and dissertation copies are in typewriter face, while others may be
from any type ofcomputer printer.
The quality of this reproduction is dependent upon the quality of the
copy submitted. Broken or indistinct print, colored or poor quality
illustrations and photographs, print bleedthrough, substandard margins,
and improper alignment can adversely affect reproduction.
In the unlikely event that the author did not send UMI a complete
manuscript and there are missing pages, these will be noted. Also, if
unauthorized copyright material had to be removed, a note will indicate
the deletion.
Oversize materials (e.g., maps, drawings, charts) are reproduced by
sectioning the original, beginning at the upper left-hand comer and
continuing from left to right in equal sections with small overlaps. Each
original is also photographed in one exposure and is included in reduced
form at the back ofthe book.
Photographs included in the original manuscript have been reproduced
xerographically in this copy. Higher quality 6” x 9” black and white
photographic prints are available for any photographs or illustrations
appearing in this copy for an additional charge. Contact UMI directly to
order.
UMIA Bell & Howell Information Company
300 North Zeeb Road, Ann Arbor MI 48106-1346 USA
313/761-4700 800/521-0600
R eproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.

R eproduced with perm ission of the copyright owner. Further reproduction prohibited without permission.

NOTE TO USERS
The original manuscript received by UMI contains indistinct,
slanted and or light print. All efforts were made to acquire
the highest quality manuscript from the author or school.
Microfilmed as received.
This reproduction is the best copy available
UMI

Using the Malcolm Baldrige National Quality Award Education Pilot Criteria for Self-
Assessment of School Districts
Presented in Partial Fulfillment o f the Requirements for the
Degree o f Doctor o f Philosophy
with a
Major in Education
in the
College o f Graduate Studies
University of Idaho
By
Sally Anderson
December, 1997
Major Professor: Dr. Cleve Taylor
Reproduced with perm ission of the copyright owner. Further reproduction prohibited without perm ission.

UMI Number: 9827869
Copyright 1998 by
Anderson, Sally C.
AH rights reserved.
UMI Microform 9827869
Copyright 1998, by UMI Company. AH rights reserved.
This microform edition is protected against unauthorized
copying under Title 17, United States Code.
UMI300 North Zeeb Road
Ann Arbor, MI 48103

Authorization to Submit Dissertation
This dissertation o f Sally Anderson, submitted for the degree o f Doctor o f Education with a
major in Education and titled, “Using the Malcom Baldrige National Quality Award
Education Pilot Criteria for Self-Assessment of School Districts” has been reviewed in final
form, as indicated by the signatures and dates given below. Permission is now granted to
submit final copies to the College o f Graduate Studies for approval.
Major Professor Date*:
■1
Date:Committee Members
Dr. Michael Tomlin
r Penny Schweibert
Date: #
Date."2
Department
Administrator
Dean, College of
Education
Dr. Roger Reynobison
/ — Date: V 9 $
Dr. .ferry T/lbhscherer
Date
Dr. Dale Gentry
Final Approval and Acceptance by the College o f Graduate Studies
V I A - — Date: _ _ 5 V / 3 / ^ /
Jeanme M. Shreeve

Abstract
The demand for improvements in education continues. Failed reform attempts,
educational fads, and poor planning designs have been cited as variables affecting the
approach to improvements in public schools. This study examines the literature o f failed
reforms, current approaches to determine the performance o f schools, and models o f
improvement based on quality theory and practices.
This study investigated the perceptions o f three types o f educators— superintendents,
principals, and teachers— regarding the performance o f their school district in seven
categories o f organizational performance. The size o f the district based on student
enrollment was used as the second independent variable to determine if there were any
significant differences in perceptions based on size o f district. An instrument was developed
using the criteria in the Malcolm Baldrige National Quality Award, 1997 version; the
Education Pilot criteria; curriculum audit standards; and accreditation standards. The study
used a proportional stratified random sampling procedure by size o f district and type of
educator. The findings were analyzed using a two-way analysis o f variance for each of the
seven categories.
The study found the reliability of the instrument to be a low o f .74 for School District
Results to a high o f .85 for Leadership. Significant differences in the perceptions of
performance o f the school districts in each o f the seven categories were found to exist
between superintendents and teachers, as well as principals and teachers. No significant
differences were found between superintendents and principals or in any category by the size
of district. The study discusses the implications o f the findings for a framework for school
improvement.

Acknowledgements
The ideas, design, and completion o f this project were the result o f many dialogues
and the collective knowledge o f many. I wish to express my sincere appreciation for the
support and guidance o f my advisor, Dr. Cleve Taylor, and the mentoring o f my committee,
Dr. Roger Reynoldson, Dr. Penny Schweibert, and Dr. Mike Tomlin.
I wish to thank Dr. Mike Friend o f the Idaho School Administrators Association and
Jim Shackleford o f the Idaho Teachers Association for their contributions o f resources and
support for this study. My sincere appreciation goes to Dr. Carolyn Keeler, Dr. Del Siegle,
and Dr. Bill Parrett for their technical expertise and recommendations. I also wish to thank
Eleanor Fisk for her assistance with the laborious task o f scanning the returned instruments
and Alice Gould, Stephanie Fox , Dawn Davis, and Chris Latter for their assistance in the
details and preparation o f this document and the defense.
My most sincere appreciation is to my husband, Mike, and our boys, A. J. and Jon,
for countless sacrifices they made so that my goals could be accomplished.

Dedication
This effort is dedicated to the three men who have taught me the most important
lessons in my life. To the memory of my father who gave me the thirst for new knowledge
and the potential to seek it; to my husband, whose love is the greatest gift o f my life and
whose commitment, support, and patience are true models for all; and to my son, A. J., who
inspires me to grow and who will always be a continual source o f pride and enlightenment.

vi
Table o f Contents
Page
Authorization to Submit Dissertation...................................................................................................... ii
Abstract.........................................................................................................................................................iii
Acknowledgements.................................................................................................................................... iv
Dedication.......................................................................................................................................................v
List o f Tables.............................................................................................................................................viii
List of Figures..............................................................................................................................................ix
Chapter 1: Introduction............................................................................................................................. 1
Background o f the Problem....................................................................................................... 1
The Effectiveness o f School Reform..........................................................................2
Statement of the Problem..............................................................................................................5
Significance o f the Problem.........................................................................................................5
Traditional Methods for Determining School Performance...............................................6
Quality Models for Organizational Effectiveness..................................................................7
Research Questions........................................................................................................................ 8
Hypotheses.......................................................................................................................................9
Limitations...................................................................................................................................... 9
Delimitations................................................................................................................................. 10
Definitions......................................................................................................................................10
Summary.........................................................................................................................................13
Chapter 2: Literature Review...................................................................................................................14
Introduction....................................................................................................................................14

Education as a System ................................................................................................................ 14
Organizational Effectiveness....................................................................................................17
Determining Effectiveness in SchoolSystems.......................................................................21
Quality Theory................................................................................................................34
Continuous Improvement in Business.....................................................................40
Summary.........................................................................................................................................59
Chapter 3: Methodology...........................................................................................................................60
Introduction....................................................................................................................................60
The Research M odel....................................................................................................................60
Instrumentation............................................................................................................................. 61
Subjects and Settings.................................................................................................................. 63
Collection o f Data........................................................................................................................ 64
Data .Analysis................................................................................................................................65
Summary.........................................................................................................................................65
Chapter 4: Findings....................................................................................................................................6 6
Introduction....................................................................................................................................6 6
Rate o f Return...............................................................................................................................67
Characteristics o f Sam ple......................................................................................................... 68
Reliability o f Performance Analysis for School Districts................................................71
Descriptive Analysis................................................................................................................... 72
Inferential Statistical A nalysis...............................................................................................115
Analysis o f “Do Not Know” Responses.............................................................................. 121
Usefulness o f the Instrument as a Tool.................................................................................122

viii
Summary......................................................................................................................................124
Chapter 5: Summary, Conclusions, and Recommendations.........................................................125
Summary......................................................................................................................................125
Conclusions.................................................................................................................................126
Recommendations.....................................................................................................................128
References................................................................................................................................................ 131
Appendix A: Instrument........................................................................................................................140
Appendix B: Panel o f Experts Used in Content Validation.........................................................175
Appendix C: Letter to Panel of Experts.............................................................................................176
Appendix D: Matrix o f Population Sample.......................................................................................177
Appendix E: Codes on the Instrument...............................................................................................178
Appendix F: Cover Letter and Directions.........................................................................................179
Appendix G: Postcard Reminder.........................................................................................................181
Appendix H: Letters o f Support..........................................................................................................182
Appendix I: Districts by Enrollment S ize.........................................................................................184

ix
List o f Tables
Page
Table 1. Factors o f Organizational Effectiveness............................................................................22
Table 2. Meta-analysis Findings of School System Evaluation Components a
Reported in the Literature....................................................................................................23
Table 3. Curriculum Audit Findings o f 67 School Districts
Between 1988 and 1994...................................................................................................... 29
Table 4. A Comparison Betweeen Teaching Theories of QualityExperts...............................40
Table 5. Baldrige National Quality Award Criteria 1997............................................................ 47
Table 6 . Validity o f the MBNQA M odel...........................................................................................48
Table 7. Accuracy o f the MBNQA W eights................................................................................... 49
Table 8 . Core Values/Concepts o f MBNQA Education Pilot1995............................................. 54
Table 9. 1995 MBNQA Educational Pilot Criteria......................................................................... 58
Table 10. Stratified Random Sample Matrix....................................................................................64
Table 11. Total Return Rates by Educator Position........................................................................67
Table 12. Frequencies and Percentages o f Returns Received by Educator Position
And District Size.................................................................................................................... 68
Table 13. Percentage of Highest Degree and Time in Position by Size and Position...........70
Table 14. Rank Order o f Combined Sam ple....................................................................................69
Table 15. Reliability o f Instrument..................................................................................................... 71
Table 16. Means by District Size and Positions Combined......................................................... 72
Table 17. Means by District Size for Districts With More Than 5,000 Students Enrolled.. 73
Table 18. Means by District Size for Districts With 4, 999 to 2,500 Students Enrolled......73

Table 19. Means by District Size for Districts With 2, 499 to 1, 000 Students Enrolled 74
Table 20. Means by District Size for Districts With 999 to 500 Students Enrolled...............74
Table 21. Means by District Size for Districts With Less Than 499 Students Enrolled....... 75
Table 22. Means by Educator Position for Superintendents.........................................................75
Table 23. Means by Education Position for Principals.................................................................. 76
Table 24. Means by Educator Position for Teachers......................................................................76
Table 25. Item Frequency and Percentage o f Response by Position and Size o f District
For Items in the Leadership Category............................................................................... 80
Table 26. Item Frequency and Percentage of Response by Position and Size o f District
For Items in the Strategic Planning Category..................................................................87
For Items in the Student Focus and Satisfaction/Stakeholder Categories............... 90
For Items in the Information and Analysis Category.................................................... 94
For Items in the Human Resource Development Category.........................................97
For Items in the Educational Process Management Category.................................. 103
For Items in the School Districts Results Category.....................................................109
Table 32. Two-Way ANOVA Leadership Construct....................................................................116
Table 33. Two-Way ANOVA Strategic Planning Construct...................................................... 117

xi
Table 34. Two-Way ANOVA Student/Stakeholder Construct..................................................118
Table 35. Two-Way ANOVA Information and Analysis Construct........................................119
Table 36. Two-Way ANOVA Human Resource/Management Construct.............................. 119
Table 37. Two-Way ANOVA Educational Process/Operational Management
Construct..................................................................................................................................120
Table 38. Two-Way ANOVA School District Results Construct.............................................121
Table 39. Chi Square for “Do Not Know” Responses..................................................................122
Table 40. Combined Percentage for Usefulness o f Instrumentation........................................123

xii
List o f Figures
Page
Figure 1. An Educational System as an Open System.....................................................................16
Figure 2. A Quality Systems Model for Performance Improvement.......................................129

1
Chapter 1
Introduction
Background of the Problem
The performance o f schools is currently determined by a multitude o f indicators
based on political, traditional, and institutional influences. Public opinion for the
performance o f the complete system is often based on one or more o f these indicators
(Bracey, 1997; Bushweller, 1996; Elam, Lowell & Gallup, 1996). Although improvements
are occurring in many o f the nation’s schools, results are still anecdotal, isolated, and far
from replicable (Fullan, 1993). Public criticism still abounds and the perception o f inferior
quality and poor performance remains (Bushweller, 1996; Hodgkinson, 1996; Houston,
1996; Huelskamp, 1993). The demands for greater accountability for publicly funded
institutions have not diminished. The lack o f evidence o f improved performance, effective
planning, and the increase spending o f public funds without discernible measures o f tangible
results have led to the demand for more business-like strategies (DeMont, 1973; Gerstner,
1995; Kearns & Doyle, 1988).
School improvement and how to achieve it continues to inspire public, political, and
professional dialogue and debate. The approach to improving public schools is as varied as
the prophets and their doctrines. Little to no sustainable improvements, public hostility, and
disenfranchised teachers are left in the wake of such well-intentioned efforts (English & Hill,
1994). When teachers from the high performing Willamette Primary School in Oregon were
asked why they thought so many schools were failing, they blamed the pursuit o f “it” (Sagor,
1995). Solving the problems in education with a one-solution approach perpetuates the
notion that “it” will remedy the problem and things will be better once we find “it.” These

2
types o f solutions are often at a visible, obvious level denying the complexity o f other
interdependent relationships and root causes (Bernhardt, 1994; Deming, 1994; Scholtes,
1995). Non-systemic interventions to improve organizations merely shift problems from one
part o f the organization to another (Senge, 1990).
The Effectiveness o f School Reform
Upon review o f current literature, factors affecting the efficacy o f reform strategies
appear to fall into three categories: (a) selection o f reform initiatives, (b) implementation o f
reform initiatives, and (c) the improvement or change strategy selected.
The 1960s saw a multitude of reform initiatives influenced significantly by a national
concern that American education was falling behind foreign accomplishments and the civil
rights movement (Fullan, 1993). Solutions were often superficial, quick-fix remedies made
impatiently as a result of various pressures facing the decision makers or educational fads
(Fields, 1994; Fullan, 1992). The result was often— and still continues to be— an abundance
of disjointed, incomplete improvement initiatives (Fullan, 1997). The presumption that
developing innovations on a national scale would lead to widespread adoption appeared to be
flawed (Fullan, 1993).
Flawed implementation is another source o f much discussion in the literature of
educational reform. Berman and McLaughlin (1977) did a comprehensive study of programs
that were federally funded. They found many examples o f failed implementation which
included failure to take into account local nuances and capacity, desire for additional funds
for political reasons rather than educational reasons, and the presumption that innovations are
implemented one at a time contrary to the reality o f schools. Another perspective offered by
Fullan and Miles (1992) is the misunderstanding o f resistance. They argue that issues of

3
effective implementation and communication strategy are often at the heart o f the reason why
reform fails rather than personal attitudes and resistance (Fullan et al, 1992).
A third reason for failed reform reflected in the literature is related to the absence o f a
specific change strategy. It has been found that often each stakeholder o f education has a
different, and usually faulty, belief about how change occurs (Fullan, 1993; Fullan et al.
1992; Hargreaves, 1997). This results in confusion and conflict during both design and
implementation phases. Denial o f the complexity o f problems and solutions is often
observed. Critics o f past reform efforts advocate for three things: (a) greater recognition of
the complexity o f the educational system; (b) deeper second order changes in the
organization; and (c) the need to be created, designed, and implemented by those
knowledgeable o f the institution (Fields, 1994; Hargreaves, 1997; O’Neil, 1995; Sarason,
1990; Wagner, 1993). Reform efforts, particularly those driven through mandated practices
contingent on state or federal dollars, often result in symbols o f improvement over substance
(Berman, 1977; Wagner, 1993).
During the past decade, most people involved in the reform o f education have come
to advocate a systemic perspective (Fullan, 1992; Timpane & Reich, 1997). The resurgence
o f interest in systems theory applications is currently resulting in a heightened attention to
and recognition o f the complexity of organizations, particularly educational institutions. A
central principle underlying systems thinking is that structure influences behavior (Deming,
1986; Patterson, Purkey & Parker, 1986; Senge, 1990). The structure, therefore, used to
initiate, conduct, and evaluate an improvement process is related to the potential
effectiveness o f each specific solution deployed.

4
Sarason (1990) proposes two basic premises to influence the design and
implementation o f solutions in school reform efforts. The first is the presence o f a
conceptual framework that recognizes the relatedness o f human behavior. The second is a
thorough understanding of the context o f that improvement. Reformers to date have been
criticized for not having an implicit theory about how to achieve change, and they do not
always recognize the influence o f the intractability o f the system (Fullan & Miles, 1992;
Sarason, 1990). Ignoring these two factors can result in an approach that seeks the cure and
ignores the diagnoses. Focusing on doing the right thing, over doing in the right way, can
result in using the means as the end (Bennis, 1976).
In K-12 educational systems, the development o f school improvement plans often
becomes a substitute for results (Sergiovanni, 1992). A focus on the outcomes or results of
education have rarely been operationalized (Schmoker, 1996). Educators often resist
confronting the results and using them to make decisions for school improvement (Bernhardt,
1994; Schmoker, 1996). Schools are traditionally limited in their use o f information and
have little need to depend on systematic feedback from a variety of their customers
(Bernhardt, 1994; Consortium on Productivity in Schools, 1995; Schmoker. 1996). The
capacity o f data and information to reveal strengths, weaknesses, successes, and failures are
threatening to educators particularly in a political context (Schmoker, 1996). Schmoker
(1996) further states that schools are too poorly organized to see the connection between
effort and outcomes.
The theoretical base upon which improvements are determined and made in the total
organization, or any part o f the total organization, is critically important in demonstrating
outcomes (Deming, 1991). The framework which follows from the theory results in the

5
models, methods, and tools (Skrtic, 1991). This study investigates a theory and the
development and use o f a framework for a process o f organizational analysis o f its
performance.
Statement o f the Problem
The process o f improving school performance often lacks an organized strategy,
processes for decision-making, deployment o f those decisions, and a mechanism for
evaluating results o f those strategies (Bernhardt, 1994; Fullan & Miles, 1992). The manner
in which schools think about the school improvement process determines their ability to
deploy it successfully (Bernhardt, 1994). The current methods o f determining organizational
performance in schools, identifying the areas o f improvement, and implementing these
changes lack a conceptual framework which recognizes the relatedness o f human behavior
(Sarason, 1990). The accreditation process, once intended to be a mechanism for self-study,
has become a political formality which focuses on the surface indicators with no mechanism
for deeper improvements leading to results (Portner, 1997). Education is lacking a useful
comprehensive framework for systemic analysis o f its performance and its approach to
improvement.
Significance o f the Problem
Goodlad (1984) remarked that in order to survive, an institution must have the faith of
its clients in its usefulness and a measure o f satisfaction o f its performance. More than 10
years later, public education continues to be challenged by the many constituencies who have
similar criticisms (Bushweller, 1996; Hodgkinson, 1996; Houston, 1996; Huelskamp, 1993;
Gerstner, 1995; Kearns et al, 1988). The use o f measures of satisfaction from the customers
o f education is limited. Subsequent approaches to improving performance that have the

6
potential o f increasing satisfaction are diverse, scattered, and often politically motivated
(Consortium on Productivity in the Schools, 1995). This research explores the existing
approaches used to determine the performance o f a school district. The arguments germane
to this research have been organized into three categories: (a) the investigation o f the
traditional methods for currently determining the performance of a school district, (b) the
concept of organizational effectiveness, and (c) the potential use o f models emerging from
quality theory to analyze organizational effectiveness in school districts.
Traditional Methods For Determining School Performance
Determining the performance of education is an undertaking worthy of
in-depth analysis o f its own. It has not been established in the literature that the measures
currently used and analyzed are the appropriate indicators o f the performance o f the
educational system (Huelskamp, 1993). Traditional methods o f assessing the successes and
failures o f public education include, most typically, multiple constituency models. These
models are designed to meet standards or criteria set by various stakeholders for various
purposes (Brassard, 1993).
Traditional models include financial, management, and curricular audit procedures;
program evaluation studies; federal or state compliance reviews; and specific indicators of
student performance. Accreditation is currently the most comprehensive practice which
purports to determine the performance o f a school (Portner, 1997). The accreditation
process, once a status symbol for schools, now is viewed as a routine examine with little
relevance to school improvement (Portner, 1997). It does not, however, look
comprehensively at the entire school district since schools are accredited as singular units.
The performance o f schools, and, therefore, school systems, is often inferred by the general

7
public based on the performance o f student scores on annual measures as reported in the
media (Rotberg, 1996). Specific units or functions o f the school district are often reviewed,
as required by state and federal laws, such as financial audits, regulations o f Title I, or the
Individuals with Disabilities Act. The focus o f these processes is to determine compliance
with regulations and the need for any corrective action. Non-compliance in some cases can
mean financial penalties to the district. Curriculum audits offer a thorough process for
assessing the organization, delivery, support, and results o f the instructional process.
Specific standards have been developed and criteria are used to determine the degree of
effectiveness. Professionally trained auditors, external to the district, conduct the process
and prepare a final report. Peer reviews, such as management audits, also occur. They are
often designed by administrators to analyze specific parameters o f management.
Quality Models For Organizational Effectiveness
According to Field (1994), the National Education Association and the American
Association o f School Administrators suggest that few innovations or educational changes
stimulated from outside o f education will occur without educator commitment. Management
is responsible for the design and approach to improving the performance of the system
(Crosby, 1984). The people who understand the processes and their outcomes well enough
are school education leaders within school organizations (Fields, 1994; Sarason, 1990). In
the absence o f a foundational theory upon which to base practices and an organized approach
to accomplish the improvements, the educational leader is vulnerable to public criticism and
negligent of their duties. There is also a need to bring the practitioner into the creation and
design o f the practice (Deming, 1993, Glasser, 1992; Imai, 1986). There are ever-increasing
examples of classroom teachers who are feeling helpless against a barrage o f public criticism

8
and increasing, uncoordinated demands o f federal, state, and local officials (Bracey, 1997;
Fullan, 1997; Hargreaves, 1997).
Quality theory, practices, and tools are being used increasingly by educational and
service organizations. Educators are applying quality principles by defining the needs and
perceptions o f internal and external customers, using information to make decisions, and
designing results-oriented strategies for systemic improvement involving people in all parts
o f the organization. The application o f the Malcolm Baldrige National Quality Award,
originally designed for business, has been extended to educational institutions and offers a
framework for analysis and recognition (National Institute o f Standards and Technology,
1995).
Research Questions
The research questions posed in this study are:
1. How do educators perceive their own school district’s performance based on an
instrument designed using the Malcolm Baldrige National Quality Award Education Criteria?
2. Are there differences in these ratings based on type o f educator or size o f district?
3. Do educators find this instrument a useful tool to study these areas o f a school
district?
4. Do educators believe this instrument could be useful in determining school
improvement needs?
Hypotheses
The study will test the following null hypotheses:
Ho i: There are no significant differences in the Leadership category o f the
Performance Analysis for School Districts by type or size.

9
Hen: There are no significant differences in the Strategic Planning category of the
Performance Analysis for School Districts by type or size.
H03: There are no significant differences in the Student Focus and Student and
Stakeholder Satisfaction category o f the Performance Analysis for School Districts by type or
size.
H04: There are no significant differences in the Information and Analysis category of
the Performance Analysis for School Districts by type or size.
H05 : There are no significant differences in the Human Resource Development and
Management category o f the Performance Analysis for School Districts by type or size.
H o 6: There are no significant differences in the Educational and Operational Process
Management category o f the Performance Analysis for School Districts by type or size.
H07: There are no significant differences in the School District Results category of
the Performance Analysis for School Districts by type or size.
Limitations
The study is subject to the following limitations:
1. The study presumes a truthful response and that respondents will understand
items.
2. Responses to items are subject to personal biases, motivations, perspectives, and
experience of the respondents.
3. Responses are presumed to be independently made.
4. Respondents’ prior knowledge o f theoretical constructs behind the instrument is
unknown.

10
5. The design o f the study is not experimental; therefore no causal relationship can
be inferred.
6 . Responses will be collected through a mailed survey that decreases the probability
of 1 0 0 % participation.
7. The study presumes that meaningful analyses can be made on less than 100% o f
returned responses.
Delimitations
1. The study is limited to superintendents, principals, and classroom teachers within
Idaho, which affects generalizability of the findings to other educators outside o f Idaho.
2. The entire population will not be used. A proportional stratified random sample
will be drawn from the population o f interest. Therefore, the data realized is subject to the
limitations o f the sample.
Definitions
The following terms are used in the study or in the Performance Analysis for School
Districts instrument:
1. Approach refers to the systems in place to improve quality and customer
satisfaction (Brown, 1994).
2. Collaborative and participatory approach to management is defined as jointly
working to identify problems and determining improvements with others in the organization
who are knowledgeable, involved, and affected by any decisions made.
3. Communication processes refer to methods used to inform and seek opinions from
others.

11
4. Comparative data or benchmarking refer to an improvement process in which an
organization compares its performance against best-in-class organizations, determines how
these organizations achieved their performance levels, and uses the information to improve
its own performance (Shipley & Collins, 1997).
5. Conventional information refers to standardized and state test scores, enrollment,
attendance, dropout, discipline, and operating budget.
6. Deployment refers to the extent to which an approach has been implemented
across an organization (Brown, 1994).
7. Educational programs and services refer to all programs and services provided to
students and conducted by professional, certified personnel or by non-certified personnel
supervised by certified personnel.
8. Educational support services refer to all programs and services which support
educational programs, such as business operations, transportation, public relations,
purchasing, clerical services, legal services, volunteers, food service, records, buildings, and
grounds.
9. Expectations refer to clearly defined statements describing specific academic,
behavioral, or social criterion to measure achievement.
10. Data and information processes include the collection, management, and
dissemination o f data on enrollment, achievement, operations, and stakeholder satisfaction
that are used in evaluation and planning processes.
11. Internal communication processes refer to personnel and students within the
school district.
12. External communication processes refer to parents and community stakeholders.

12
13. Human resource area includes employee well-being, satisfaction, professional
development, work system performance, and effectiveness.
14. Human resource indicators include employee well-being, labor relations,
satisfaction, professional development, work system performance, and effectiveness.
15. Leadership refers to district-level senior administrators and board o f trustees.
16. Organizational effectiveness is a social construct referring to the quality o f an
organization that achieves the performance expected o f it (Brassard, 1993).
17. Performance refers to the results produced by the school district as illustrated by
multiple indicators.
18. Performance data includes data or information from all aspects o f the
organization, including student performance measures, enrollment, discipline, human
resources, business operations, and community.
19. Results refer to data on the performance o f the organization (Brown, 1994).
20. School district units refer to the specific schools, departments, or services o f that
school district.
21. Stakeholder refers to individuals or groups, both internal to the school (students,
all personnel) and external (parents, community members, business) which are affected by
the conditions and quality o f education and the preparedness o f graduates.
22. Student conduct indicators refer to measures of student behavior such as
disciplinary infractions, suspensions, expulsions, arrests, etc.
23. Strategic development refers to the process by which members o f an organization
clarify the purpose and develop the necessary procedures and operations to achieve a purpose
and design a strategic plan.

13
24. Suppliers refer to those businesses or individuals with which the district contracts
for specific services such as training, consulting, transportation, legal, etc.
25. Partnering processes refer to relationships between organizations, community
agencies, and businesses through which services for students and stakeholders are designed,
implemented, and provided.
26. System refers to a complex of elements in mutual interaction (Owens, 1970).
27. Total quality or continuous improvement refer to a system that elicits
organization-wide participation in planning and improving processes in order to meet or
exceed customer expectations.
28. Work systems are defined as the way in which jobs, work, and decision-making
are designed at all levels within the organization.
Summary
There are few organized approaches to the assessment o f performance in school
districts that apply an integrated analysis of the subsystems. The lack o f use o f information
to make strategic improvement decisions and a systems-based approach to assessing the
current effectiveness contributes to unsuccessful reform initiatives in education. An initial
step in any process o f examination is to determine what now exists (Goodlad. 1984). This
study seeks to determine the usefulness of an assessment process to a acquire baseline
perception of the organization’s performance as it exists today using three constituencies of
the organization.

14
Chapter 2
Literature Review
Introduction
Areas o f literature previously cited in Chapter 1 are investigated in depth in this
chapter. First, it is critical to understand the nature o f the educational institution as a system.
Defining the nature of educational subsystems and their relationship to each other in the
larger system is fundamental to understanding how to study and improve its effectiveness.
Second, organizational effectiveness theory and practice is discussed. Third, current
approaches to determining school effectiveness and issues o f accountability are reviewed.
Fourth, quality theory is discussed as foundational to understanding the emerging
applications in both business and education. Finally, the applications o f the Malcolm
Baldrige National Quality Award in business and education are described.
Education as a System
During the past decade there has been an increased attention to systems thinking. The
field o f systems thinking includes cybernetics, chaos theory, and Gestalt theory, and is
reflected in the works of Ludwig von Bertallanfy, Russell Ackoff, Gregory Bateson, and Eric
Trist (Senge et al, 1994). Ludwig von Bertalanfy recognized the relationships among several
important concepts current in biology in the 1930s (Levine & Fitzgerald, 1992). He named
the integration o f these ideas general systems theory, incorporating cybernetic concepts such
as feedback. Miller (1993) describes the theory as a philosophy of science that studyies
natural phenomena of all sorts as heterogeneous wholes composed o f multiple different but
interrelated parts rather than studying each part in isolation. Three types o f systems are
described in the literature of the biological sciences. Isolated systems are described by

15
Nicolis and Prigogine (1977) as those which do not exchange matter or energy with their
environment. Closed systems do exchange energy with their environment while open
systems exchange both energy and matter with their environment (Nicolis & Prigogine,
1977).
Systems thinking is a discipline for seeing the wholes and the pattern o f
interrelationships among key components o f a system (Eisenberg & Goodall, 1993; Owens,
1970; Senge, 1990; Senge et al, 1994). A system is a collection o f parts that interact to
function purposefully as a whole (Deming, 1986; Patterson, 1993; Senge. 1990). Inter
dependence is the primary quality o f a system. It refers both to the completeness o f the
workings o f a system in its environment and the interrelationships o f individuals that fall
within the system. These interdependent relationships between people give the organization
its culture. Process, feedback, and contingency are also components o f systems (Eisenberg &
Goodall, 1993).
From a systems perspective, a school district, like other organizations, does not exist
as an entity unto itself, yet it often behaves as one (Eisenberg & Goodall, 1993). School
districts are both open and social systems (Hoy & Miskell, 1991; Owens, 1970). A social
system is defined as an interactive, interrelated, and interdependent network o f components
and unique organizational properties that form an organized whole and function to serve
common goals (MacLellan, 1994). An open system depends on the external environment for
their continued existence, requiring resources from external inputs to the systems
(Consortium on Productivity in the Schools, 1995; Deming, 1986; Eisenberg & Goodall,
1993; Owens, 1970; Senge, 19903). Figure 1 presents a school district as an open system.

16
People []""
Money
Support
Resources
INPUTS OUTPUT
Education
System
$$CUSTOMERS
Higher Education
Taxpayers
Employers
Educated students who
can continue to learn
Figure 1. An Educational System as an Open System.
In open systems, groups outside the system affect the system’s survival and their
ability to change. School districts take in political, financial, and human resources and use
them to create a service. This service results in a product to the surrounding environment of
the workplace, higher education, and community. Open systems theory emphasizes the
dynamic aspects o f organization; that is, the movement in one part leads in predictable
fashion to movement in other parts. They are in a constant state o f flux because they are
open to inputs from the environment (Katz & Kahn, 1978).
School districts have also been described as loosely coupled systems (Weick, 1976).
Weick (1976) explains that the use of the term intends to convey the image that coupled
events are responsive but that each event also preserves its own identity and some evidence
of its physical separateness. There is usually lack o f clarity, coordination, and articulation
between and among subsystems within the larger system, despite their interdependence.
Such systems often are organizations in which accountability and interdependence between
subsystems are low and autonomy is high (Deer, 1976; Fullan, 1980). Subsystems are
purposely not closely connected and do little to control each other’s activities. They tend to

17
respond by shutting out environmental threats and increase the sense o f efficacy and
autonomy o f its members.
Theories o f bureaucracy from which schools continue to be organized, have paid little
attention to the organization’s dependency on internal and external environments (Deming,
1986; Eisenberg & Goodall, 1993). The task o f teaching was viewed as clearly understood,
routine, and predictable (Katz & Kahn, 1978; Owens, 1970). This mechanistic approach to
organizing work in schools is considered to be efficient. This structure does not work when
the environment is uncertain and, in fact, interferes with the organization’s ability to be
adaptive to its inputs (Consortium o f Productivity in Schools, 1995). The influence of
systems theory results in emphasis on inputs, processes, how the processes interact,
information flow and feedback, management o f relationships, and outputs (Deming, 1986;
Eisenberg & Goodall, 1993).
Organizational Effectiveness
The body of knowledge o f organizational theory, behavior, and the pursuit of a model
to determine organizational effectiveness is substantial, confusing, and often in conflict
(Brassard, 1993; Georgopoulos, 1957; Zammuto, 1982). Organizational effectiveness has
been defined in the literature in a variety o f ways. Attempting to define effectiveness,
develop criteria, and apply them to a variety o f organizations continues to be noted in the
literature (Brassard, 1993; Cameron, 1980; Georgopoulos, 1957; Zammuto, 1982).
Yuchtman (1967) points out two assumptions that are either implicitly or explicitly made: (a)
Complex organizations have an ultimate goal or function, and (b) the ultimate goal can be
identified empirically and progress toward it measured. How organizational effectiveness is

18
defined is related to the theoretical model from which it was developed. Three models
emerge in the literature (Brassard, 1993; Zammuto, 1982).
As early as the 1930s the emergence o f a goal-based approaches to organizational
effectiveness can be seen (Brassard, 1993; Cameron, 1980; Zammuto, 1982). These models
are often referred to as rational models and are functional rather than conceptual
(Georgopoulos, 1957). Organizational effectiveness is seen as the degree o f achievement of
multiple goals or the degree o f congruence between organizational goals and observable
outcomes (Zammuto, 1982). The focus o f the rational organization is goal orientation
(Cameron, 1980; Patterson, Purkey & Parker, 1986). The design, articulation, and
achievement o f goals are emphasized in the organizations applying this model. The
assumptions in this model are that goals remain stable over time, goals are determined by the
leaders o f the organization, and goals become translated into objectives within the sub-units
of the organization (Patterson et al, 1986). The organization is seen as an entity rationally
structured in order to achieve the goals to which it ascribes. The goals are typically created
to help the organization achieve its expected performance.
The focus o f evaluating effectiveness from this model is on the outputs produced by
the attainment o f goals (Cameron, 1980). The development o f efficiency-related criteria to
insure the accomplishment o f goals is often designed to influence the use o f resources to
achieve optimal performance, productivity, and profits for the organization (Bressard, 1993).
This model led to such practices as management by objectives that remained popular through
the 1970s. The focus o f management in this model is the accomplishment o f the
organization’s goals (Hersey & Blanchard, 1982). The emphasis o f management within this
model is setting goals and objectives that are accomplished by motivating and controlling

19
others in the organization to carry them out. In practice, however, organizations often do not
reflect these goals in their daily activities, either at a micro or macro level (Brassard 1993).
There are limitations to a goals approach pointed out in the literature (Cameron, 1980;
Etizoni, 1960; Katz & Kahn, 1978). Success may be overlooked if there is no goal to
measure it. Goals may be too low, misplaced, or harmful long term. Goals are usually
expressed as idealized states and are often not realistically assessed. The nature and effects
o f the social systems from which goals emerge often are not considered in the attainment o f
goals. The goals model may be useful when organizational goals are clear, consensual, and
measurable (Cameron, 1980; Patterson, Purkey & Parker, 1986). The criteria for determining
effectiveness then become unique to that organization and its goals.
Systems-based approaches emerged during the 1950s, according to Owens (1970) and
Zammuto (1982). These models draw on the emerging body of general systems theory
discussed initially in this chapter. Applying this theory, organizational effectiveness is then
viewed as the extent to which an organization as a social system fulfills its objectives without
incapacitating its means and resources and without placing a strain upon its members
(Zammuto, 1982). W. Edwards Deming believed that systems are developed to perform
repetitive tasks (Deming, 1982). Most problems within organizations, he believed, came
from sub-optimization of that system, meaning the system was performing these tasks below
their capability. The inconsistencies and contradictions that become apparent upon analysis
o f the system can be used to detect and isolate the flaws of that system (Bradley, 1993).
Other models appearing during the 1970s are referred to as the multiple constituent
definitions o f effectiveness. The organization is effective insofar as it meets the expectations
o f actors associated with it in one way or another and who try to promote their objectives and

20
interests (Brassard, 1993; Cameron, 1980). An organization is effective insofar as the
majority o f those participating in it perceive that they can use it to satisfy their interests. This
model stresses the importance o f satisfying the expectations o f the actors who agree to
support the organization and who influence its ability to obtain the resources it needs and
conserve its legitimacy (Brassard, 1993). This model acknowledges up front the influence
that these constituents have on the organization. Related to these models are those
approaches driven by the requirements o f external organizations such as accreditation
agencies, laws, and regulations (Brassard, 1993).
Dubin (1976) pointed out that organizational effectiveness has a different meaning,
depending on whether the organization is viewed from the outside or inside. The inside
perspective o f an organization tends to be a traditional managerial viewpoint which
emphasizes return on investment and efficient use o f resources. The perspective from the
outside evaluates the output of the organization relative to its contribution to the environment
or the context outside the organization. Dubin (1976) further points out that there is no
correlation relationship between these two perspectives. In fact, he says, they are worlds
apart and cannot be reconciled. “We must face squarely the fact that organizations live under
conflicting demands regarding their effectiveness” (Dubin, 1976, p. 8). Bass (1952)
suggested the criterion of organizational success needed to be expanded to include measures
relevant to employees, society as a whole, and the organization’s management. He suggested
organizational performance should be assessed based on: (a) the degree to which an
organization’s performance was profitable and productive, (b) the degree to which an
organization was of value to its employees, (c) the degree to which an organization and its
members were of value to society. Campbell et al (1974) found over 25 different variables

21
that were used as measures o f effectiveness in organizations prior to 1973 (see Table 1). The
most commonly recurring criteria were adaptability/flexibility, productivity, and satisfaction.
Determining Effectiveness in School Systems
MacLellan (1994) did a meta-analysis o f organizational effectiveness criteria used in
school systems. He found fifteen criteria: goals, environment, leadership, structure, work
force, interaction, process, decision-making, workplace, culture, change, communication,
curriculum, accountability and politics. The studies used ranged from 1967 through 1991.
Determining the effectiveness of schools is even more elusive than for other
organizations. Schools, universities, and colleges have been referred to as organized anarchy
in the literature o f organizational study (Cameron, 1980). Some typical characteristics are:
1. Goals are ill-defined, complex, changing, and often contradictory. Goals of some
sub-units may be unrelated to the broader organizational goals.
2. There is often no connection in the way work is done and the outcome.
3. More than one strategy can produce the same outcome.
4. There is little or no feedback from the output to the input.
5. Sub-units are not tightly connected, so it is easier to ignore outside influences.
6. Widely differing criteria of success may be operating simultaneously in various
parts of the organization.
There is often an ambiguous connection between the organizational structure and the
activities o f the organization. It is typical to find rigid structures and hierarchies imposed
upon loose, fuzzy processes.

22
Table 1. Factors o f Organizational Effectiveness
Overall effectiveness o f the organization Productivity
Efficiency Profit
Quality Accidents
Growth Absenteeism
Turnover
Motivation
Control
Flexibility/adaptation
Role and norm compliance
Readiness
Utilization o f environment
Evaluations by external entities
Internalization o f organizational goals
Satisfaction
Morale
Conflict/cohesion
Goal consensus
Managerial task skills
Managerial interpersonal skills
Managerial management communication
Stability
Value o f human resources
Note: From The Measurement o f Organizational Effectiveness: A Review o f Relevant
Research atid Opinion (pages 39-40), by J. P. Campbell, E. A. Brownas, N. G. Peterson and
M. D. Dunnette, 1974, San Diego: Naval Personnel Research.
Cameron (1980) makes the point that none o f the described models of organizational
effectiveness will work for organized anarchy. Criteria o f effectiveness are usually vague
and ambiguous, making organizational goals difficult to measure and not necessarily agreed
upon by all sub-units. There is often no feedback loop between outputs and inputs, making
the systems model an unnatural fit. Cameron (1980) suggests that the multiple constituencies
model may be the most appropriate for the organized anarchy. The demands o f the
constituencies, once defined, can be assessed on the degree to which they are met.

23
Although there is substantial information regarding evaluation of schools and school
programs in the literature, there is little consistency in the approaches and parameters
evaluated. Eight components are reported by Nowaiowski (1985). They include business
and finance, curriculum and instruction, policy, planning and evaluation, pupil personnel
services, personnel, school-community relations, and school management. MacLellan (1994)
ranked components o f school systems evaluated in the literature. Table 2 illustrates his
findings.
Table 2. Meta-analysis Findings of School System Evaluation Components by Rank as
Reported in the Literature.
1. Goals 8. Decision-making
2. Environment 9. Work place
3. Leadership 10. Culture
4. Structure 11. Change
5. Workforce 12. Communication
6. Interaction 13. Curriculum
7. Process
Note: From “Towards a New Approach for School System Evaluation,” (Page 159), by
David MacLellan, 1994. (Doctoral dissertation, Dalhousie University, Nova Scotia).
DisseNational Abstracts International.
There are three methodologies that appear to be represented in a substantial manner in
the literature: (a) evaluation research, (b) curriculum audits, and (c) effective schools
research. The researcher has also included a discussion o f the accreditation process in Idaho.
The discussion o f the effectiveness of schools is often linked with discussions of
accountability or to performance of student learning. Although both factors are relevant to

24
this study, the researcher focused on models that were using multiple indicators of
effectiveness.
A key deficit in most educational systems, which is all to frequently pointed out by
the critics of public education, is the lack of effective evaluation (Worthen, 1987). The
public demand for accountability has made many educators fearful o f the concept. DeMont
and DeMont (1973) suggested three improvements to the process o f demonstrating
accountability: (a) an increased focus on the outputs o f education, (b) the production of more
effective evaluative and research models, and (c) the inclusion o f non-educators in the
decision-making process. They suggest that an accountability model be a comprehensive
plan for problem solving aimed at improving educational practice. The requirements of this
model include: (a) the designation o f the persons responsible for the program operation.
(b) conducting an internal program review, (c) conducting an external program review, and
(d) use o f the results to diagnose needs and prescribe action.
Evaluation research has been a tool used frequently in public schools to make
judgments about the merit, value, or worth of educational programs (Gall, Borg & Gall.
1996). They are most often used to determine the effectiveness o f specific programs,
benefits to cost ratios, or areas for improvement. Formal evaluation consists of systematic
efforts, using qualitative and/or quantitative designs to define criteria and obtain accurate
information (Worthen, 1987). Formal evaluation studies are often done as a basis for
decision-making and policy formation, to evaluate curricula, monitor expenditure of public
funds, or improve educational programs (Worthen, 1987). Worthen (1987) notes that many
evaluation studies do not lead to significant improvements in school programs. He cites
several reasons including inadequacies o f research design, the use o f evaluation information.

25
and the view o f evaluation as a discrete study rather than a system o f self-renewal (Worthen,
1987/ The Joint Committee on Standards for Educational Evaluation (1994) developed
standards designed for use in judging quality o f educational evaluation. These standards
cover criteria involving the utility o f the evaluation, its usefulness to the persons involved,
feasibility o f the design to the setting, legal and ethical factors, and the extent to which the
study yields valid, reliable, and comprehensive information for making judgements.
Guba (1981) outlined four major models o f educational evaluation: (a) objectives,
(b) outcomes, (c) effects, and (d) audience concerns. Evaluation approaches that are based
on specific goals and objectives assess the congruence between the standard or the goal and
the performance (Provus, 1971). In a discrepancy-based model o f evaluation, standards are
defined and developed, the performance is assessed, the discrepancy is determined, there is
feedback to the decision-makers, and there is a decision. The critical point in this model is
the establishment o f a standard and assessment against that standard. The C. I. P. P.
(Context, Input, Process, Product) model is a decision-making approach relying on
generation o f information to be used in making decisions (Stufflebeam, 1983). The context
provides an illustration o f the needs and goals, input on how resources and procedures are
used to reach goals. The process focuses on any defects in the implementation of those goals
and products and the measurement of the outcomes. Scriven (1973) proposed the consumer-
oriented model that included establishing standards or indicators; comparing effects to
benefits and costs; and making judgements about change, use, and choice. The focus in this
model is the judgement o f merit or worth. The countenance model— later called the
responsive mode— distinguishes three phases: (a) antecedents, (b) transactions, and

26
(c) outcomes (Stake, 1967). This model relies on an informal framework in which
observation, judgement, and data matrices are emphasized.
Curriculum audits provide another source o f evaluating organizational effectiveness
for schools. Curriculum management audits were first offered by the accounting firm of
Peat, Marwick, Mitchell and Company where a partner, Fenwick English, adapted it from the
financial audit process (Vertiz, 1995). He brought the service to the American Association o f
School Administrators which created the National Curriculum Audit Center. The Center
trains curriculum auditors and contracts with school districts. The first audit was done in
1979 in the Columbus Public Schools in Ohio. As o f April 1995, curriculum audits had been
performed in nearly 100 school districts in the United States and two foreign countries
(Vertiz & Bates, 1995). According to Vertiz (1995), the audit became an important data
source in state take-over o f school systems in New Jersey and Kentucky. It is based upon the
concepts of effective instruction, curricular design, and delivery. The audit is designed to
determine the extent to which a sound, valid, and operational system o f curriculum
management is implemented (Vertiz & Bates, 1995). According to Vertiz and Bates,
curricular quality control requires: (a) a written curriculum in a clear, translatable form for
application by teachers in classrooms or related instructional settings; (b) a taught curriculum
which is shaped by, and is interactive with, written curriculum; and (c) a tested curriculum
which includes the tasks, concepts, and skills o f pupil learning that are linked to both the
taught and written curricula.

27
English (1988) described the five standards he created for the auditing process:
1. The school district is able to demonstrate its control o f resources, programs, and
personnel. Control refers to the system’s ability to channel and focus its resources toward the
achievement o f its goals and its mission (Kemen, 1993). Auditors look for indicators that
demonstrate linkages between the board, central management, and the instructional process
(English, 1988).
2. The school district has established clear and valid objectives for students.
Auditors examine board policy, administrative procedures, courses o f study, and scope and
sequence of curriculum (English, 1988).
3. The school district has documentation explaining how its programs have been
developed, implemented, and conducted. The district must demonstrate clear and operational
linkages between all layers o f the system. Auditors look for alignment between policy,
curriculum, instruction, materials, and assessment (English, 1988).
4. The school district uses the results from the district designed or adopted
assessments to adjust, improve, or terminate ineffective practices. Auditors evaluate the
extent to which the district collects data to evaluate its performance. The data should reflect
its goals, provide usable information that should be used to adjust, or improve district goals
(English, 1988).
5. The school district has been able to improve productivity. Productivity is the
relationship between the inputs and the cost o f obtaining any given level o f outputs (English,
1988).
Each standard encompasses numerous criteria. These criteria are evaluated through
document reviews, interviews with the board and professionals, and observations by trained

28
auditors who are school administrators external to the organization. The data is then
triangulated according to agreed upon conditions. Table 3 illustrates specific criteria and the
findings o f a study o f audits conducted between 1988 and 1994 by Vertiz and Bates (1995).
The authors conclude that the majority o f school districts who participated in the audit
process were deficient in major management structures and functions that pertain to the
design and delivery o f curriculum. The investigators found that 90% or more o f the findings
were deficient in 80% o f the areas investigated.
Kamen (1993) found that the extent o f implementation o f the audit recommendations
is dramatically affected by the nature o f the audit selection method. When the audit is
voluntarily selected by a district, there is a high level o f implementation. There are positive
effects generally as demonstrated by greater empowerment of all personnel and a tendency
towards a systems perspective. Management processes appeared to improve. Results
suggest that there is significantly less implementation o f recommendations when the process
is mandated. Under some conditions, it can become a political battleground with resistance,
denial, and defensiveness.

29
Table 3. Curriculum Audit Findings o f 67 School Districts Between 1988 and 1994.
Standard number: Criteria Strong rating Deficient rating
1 : Policy design 6% 94%
I : Policy implementation 0% 100%
1 : Planning design 10% 90%
1 : Planning implementation 10% 90%
I : Organizational structure 5% 95%
1 : Organizational implementation 27% 73%
1 : Personnel practices 0% 100%
1 : Personnel supervision and supervision 14% 86%
2 : Instructional goals and objectives 6% 94%
2 : Curriculum scope 22% 78%
2 : Curriculum guide: design 2% 98%
2 : Curriculum guide: delivery 0% 100%
2 : Curriculum management structure 3% 97%
3 : Internal consistency 3% 97%
3 : Equity: design 5% 95%
3 : Equity: implementation 7% 93%
3 : Monitoring practices 4% 96%
3 : Staff Development: design 2% 98%
3 : Staff development: delivery 0% 100%
3 : Articulation and coordination 2% 98%
(table continues)

30
Table 3, cont’d. Curriculum Audit Findings o f 67 School Districts Between 1988 and 1994.
Standard number: Criteria Strong rating Deficient rating
Testing program: scope 2% 98%
Testing program: quality 3% 97%
Use of assessment data 2% 98%
Use o f program evaluation data 0% 100%
Curriculum-driven budget 0% 100%
Cost effectiveness 4% 96%
Organizational improvement 0% 100%
Facilities 39% 61%
School climate 83% 17%
Support system functioning 30% 70%
Note: From The Curriculum Management Audit: Revelations About Our School (1995), by
Virginia Vertiz and Glynn Bates. Paper delivered to the American Education Research
Association, Division B.
Effective schools research offers a set o f criteria for determining organizational
performance. Effective schools have been described by several parameters. Effective
schools add value through their services, high evaluations from students, high expectations
and high norms of achievement, strong leadership, collaborative decision-making, clear
goals, system wide culture, safe environment, and a dedicated workforce (Mann, 1976,
Purkey & Smith, 1982). Seven characteristics emerged from the body o f literature known as
effective schools research (Edmonds, 1980). They include (a) strong, instructional
leadership; (b) a safe, orderly climate; (c) high expectations for achievement; (d) emphasis on

31
basic skills; (e) continual monitoring o f progress; (f) goals that are clear and understood; and
(g) culture.
Research began in the mid-1970s to determine why some schools were effective and
others were not. The research of Ron Edmonds and Lawrence Lezotte began in large urban
schools. Their study resulted in the identification o f correlates present in effective schools
(Edmonds, 1979; Lezotte & Bancroft, 1985). Strong administrative leadership was found to
exist, particularly focused on planning, supporting and monitoring the instructional process.
There were high expectations for all students and the staff o f the building. A positive school
climate existed in the building as evidenced by a sense o f pride and community. There was a
focus on the instructional program in the total school with emphasis on training in teaching
practices. Finally, there was a thorough assessment process allowing for continual
monitoring o f student progress. Numerous improvement strategies followed that focused on
developing the specific correlates in schools. Stefanich (1983) pointed out that much of the
impetus for these applications was based on intuitive rationale rather than hard data. Lezotte
(1989) has since integrated quality theory into his approach to school improvement, citing
such precepts as the importance of an attitude o f continuous improvement, a deliberate
change strategy, and attention to all parts of the system. Some independent contractors using
the correlates o f effective schools as the standard have created an audit-type process.
Each state has an accreditation process usually affiliated with a regional accrediting
organization (Portner, 1997). In Idaho the purpose o f accreditation is to help schools achieve
the required Standards for Idaho Schools and enhance school improvement (Idaho State
Department of Education, 1996). There are four options for how Idaho schools seek
accreditation. They may choose one o f the following options:

32
1. The Idaho Elementary/Secondary Accreditation Standards.
2. The Northwest Accreditation Standards.
3. The Idaho School Accreditation School Improvement Model.
4. An alternative school improvement plan.
Regardless o f the option selected, the school must demonstrate the standards defined
and the components o f thoroughness as specified by Idaho Code 33-119. A thorough system
o f education has been defined in Idaho code as one in which:
1. A safe environment conducive to learning is provided.
2. Educators are empowered to maintain classroom discipline.
3. The basic values o f honesty, self-discipline, unselfishness, respect for authority,
and the central importance o f work are emphasized.
4. The skills necessary to communicate effectively are taught.
5. A basic curriculum necessary to enable students to enter academic or vocational
post-secondary educational programs is provided.
6. The skills necessary for students to enter the workforce are taught.
7. The students are introduced to current technology.
8. The importance o f student’s acquiring the skills to enable them to be responsible
citizens o f their home, schools, communities, state, and nation is emphasized.
Regardless o f the option selected, schools must demonstrate on an annual basis the
five required standards:
Standard I: Philosophy/Mission, Vision, Polices: School philosophy and policies
need to be aligned with thoroughness legislation.

33
Standard II: Personnel and Certification: All educators o f Idaho students must be
certified as specified in the State Board o f Education Rules for the Public Schools o f Idaho.
Standard III: Curriculum/Instruction/School Improvement: This standard is defined
in the thoroughness legislation.
Standard IV: Accountability/Assessments/Measures: Schools must establish
standards for all grade levels and high school exiting standards for graduation, participate in
statewide testing programs, have written plans to reduce dropouts, and report on student
attendance.
Standard V: Safe Learning Environment: Schools must have safe facilities. Each
school must have a comprehensive, district-wide policy and procedure in place encompassing
safe environment and discipline.
There are specific additional standards for each level— elementary, middle, and
secondary. Each standard has specific criteria to which deviation points are assigned.
Schools are accredited annually according to ratings determined by points. If schools receive
a status of not approved for more than one consecutive year, state funds can be withheld and
a report to the public is made. The Northwest accreditation process involves a self-study for
initial accreditation involving staff, students, and community (NASC, 1996). The
accreditation process runs on a ten-year cycle involving a self-study during the ninth year of
the process.
What is unclear in the literature is how the information from any method of
determining organizational effectiveness is used. Brassard (1993) cautions against the need
to compare the performance o f organizations or to identify characteristics o f those that are
effective. Having criteria o f effectiveness reinforces the notion that: (a) organizations

34
possess an inherent rationality and (b) criteria become requirements which are imposed often
independent o f their purpose. He makes the point that criteria adopted must define the
performance that the organization must achieve if it is to be useful. Hannan and Freeman
(1977) argue that inter-organizational comparisons can not be accomplished because there is
no way for scientific analysis o f comparative organizational effectiveness.
The above review o f approaches to the overwhelming task o f determining
organizational effectiveness illustrates the varied strategies used in the past and present.
Such performance assessments are done for different reason. There is little discussion in the
literature regarding the process o f evaluation of organizational self-study for the ultimate
purpose of improvement. There is an increasing interest in action research or practitioner
based research done by the practitioners within their own site as a reflective process of
investigation (Anderson et al, 1994). Practitioner research is best done as a collaborative
effort to accomplish multiple perspectives for the purposes o f taking actions in a specific
situation. Zammuto (1982) points out that it is useful to remember that organizations are
social inventions created to satisfy human needs. These needs influence how people evaluate
the effectiveness of organizational performance based on their experience with organizations
and the impact of that performance on them or their preferences. The purpose o f assessment
in anything is to determine the performance and then improve it.
Quality Theory
The importance o f theory as it relates to the areas cited above is explored since from
theory, assumptions, models, practices, and tools emerge (Skrtic, 1991). Many companies
today are using total quality theory or continuous improvement theory as both a conceptual
framework and operationally (Walton, 1990). Total quality or continuous improvement is an

35
approach to organizational development that has both historic roots and evolving tenets. It
involves both reflective and active components for organizational development. It has been
defined as a people-focused management system that aims at continual increase o f customer
satisfaction at continually lower real cost (Crosby, 1984; Deming, 1986; Imai, 1986). This
theoretical model is a systems approach to organizational improvement, meaning that
improvements should be made with the whole organization in mind (Deming, 1986, 1994;
Senge, 1990). The terms total quality control and total quality management were coined by
Fiegenbaum (1983). He defined total quality control as “an effective system for integrating
quality development, quality maintenance, and quality improvement efforts o f various groups
in an organization so as to enable marketing, engineering, production, and service at the most
economical levels to allow for full customer satisfaction” (Fiegenbaum, 1983, p. 823). He
used the term total to mean a systems approach to achieve excellence. He defined quality in
terms o f the specific requirements o f the customer.
Japanese management theory has influenced quality theory in the Western world
significantly. Referred to as Kaizen in Japan, it is the single most important concept
influencing Japanese management (Imai, 1986). The Kaizen philosophy means on-going
improvement through the involvement o f everyone, in all aspects o f life. Imai remarks, “I
came to the conclusion that the key difference between how change is understood in Japan
and how it is viewed in the West lies in the Kaizen concept; a concept that is so natural and
obvious to many Japanese managers that they often do not realize that they possess it!”
(Imai, 1986, p. 3). He concludes that this concept is either very weak or non-existent in
American and European business based on his many years o f studying the differences.

36
There are consistent fundamental principles in quality theory, which emerge upon
review o f the literature by those who are credited as being the major experts in the quality.
The researcher will focus on these principles rather than an in-depth analysis o f the historical
perspectives o f quality theory. Upon review o f the literature, it is clear that quality theory
has emerged from a variety o f historical management approaches, economic contexts o f the
times, cross-cultural influences o f both East and West, and an ever-increasing body of
knowledge that is evolving through practice (Crosby, 1984; Danne, 1991; Deming, 1986).
W. Edwards Deming, often considered the “father of quality,” developed a theory of
profound knowledge that incorporates the major tenets o f quality theory (Deming, 1986,
1989, 1994). He believed that not only skills, but also knowledge about management was
paramount. Deming (1989) stated, “hard work and best efforts, put forth without guidance of
profound knowledge, leads to ruin in the world that we are in today. There is no substitute
for knowledge” (Deming, 1984, p. 10). The system of profound knowledge includes four
principles, each related and interacting with the other.
The first principle is appreciation for a system, which Deming defined as “a network
o f interdependent components that work together to try to accomplish the aim of the system”
(Deming, 1984, p. 50). He stressed the interdependencies within a system and the necessity
o f cooperation among the parts. The greater the independence between the components, the
greater the need for communication and cooperation between them. The system needs to
have an aim that is clear to all in the organization. Without this clear purpose, says Deming.
the aim becomes a value-judgement made on individual bases (Deming, 1984). Deming
often used the example of a good orchestra to illustrate a well-optimized system. “The
players are not there to play solos as prima donnas, to catch the ear o f the listener. They are

37
there to support each other. They need not be the best players in the country.” (Deming,
1984, p. 15). According to Deming, management o f a system is action based on prediction.
The prediction needs to be rational and based on the information that the system teaches
people in the organization. Therefore, the performance o f any part o f the system must be
judged in relationship to other parts and the aim of the system.
A second element is knowledge o f variation or statistical theory. Deming believed
that without statistical analysis methods, attempts to improve a process would be hit or miss.
Understanding that variation will always exist in all components o f a system— people,
processes, results— is fundamental. He called for an understanding o f the capability of a
process. Developing stable processes— which means the process is in a state o f statistical
control— is the goal in determining a system’s capability. He makes the distinction between
the types o f variation, special cause, and common cause. Common cause he defines as the
variations that occur by chance and can be attributable to a system Special causes, on the
other hand, are caused by events outside of a system. Deming felt that these were important
to know before one attempted to work on a system (Deming, 1986, 1989). If these
distinctions are not understood, he suggested, mistakes can be made that are costly and
ineffective.
The prevention of errors and nonconformance to specifications are key principles
resulting from the knowledge o f variation (Crosby, 1984; Deming, 1986). Philip Crosby, a
recognized quality expert, invented the term zero defects which he defined as no acceptable
rate of defects for products or services that do not meet customer’s requirements (Crosby,
1984). The emphasis is on prevention, rather inspection or the process of detecting the good
and the bad.

38
Deming’s third principle builds on the need to use the system and its variation to
generate what he called the theory o f knowledge (Deming, 1986, 1989, 1994). He believed
that good intentions were not enough for management. Managers, he insisted, need to
continually build knowledge and theories based on that knowledge. Deming believed that
this was the only basis for management’s ability to predict. When processes are in the state
o f statistical control, statistical theory can assist in prediction. Theories then emerge from
this knowledge. Theories, professed Deming, are necessary to generate questions. Without
questions, there may only be examples o f successes, and if these are duplicated under the
pretense o f a solution, they can lead to failure. Continual approach o f narrow solutions can
lead to more and more of the solution (Senge, 1990). Theory is critical in optimizing a
system which can meet the customer’s expectations the first time (Deming, 1986, 1989;
Crosby, 1984).
Joseph Juran, another quality expert, extended Deming’s beliefs as they pertained to
knowledge-based decisions to the role o f management (Juran, 1988). He believed that it was
the responsibility o f top management to lead the company through massive training in
quality. Juran placed an emphasis on planning, customer satisfaction, and the use o f data
collection and analysis and has been credited with being the first to address the broader
issues o f management as they relate to quality (Danne, 1991; Miller, 1993).
The fourth principle o f profound knowledge is psychology. Deming felt that this
body o f knowledge was critical in the interaction between people and circumstances, the
interaction between people, and the interaction between people and the system (Deming,
1986, 1989). He emphasized the importance o f leaders in recognizing the differences in
people and using these differences to optimize the system. Recognition of differences in how

39
people leam, how they work, and how they relate to each other is an additional factor that a
manager should understand. Leaders are obligated to make changes in the system which will
result in improvement (Crosby, 1984; Deming, 1986).
Critical to this system o f profound knowledge are the following:
1. People have an innate need for self-esteem and respect.
2. Circumstances can either provide or deny people opportunities for dignity and
self-esteem.
3. Management that denies opportunities for dignity and self-esteem will smother
intrinsic motivation.
4. Some extrinsic motivators rob employees o f dignity and self-esteem.
5. Management should recognize the innate inclination o f people to leam and invent.
Deming believed that new systems o f rewards needed to be established to restore respect for
the individual and release the potential o f human resources (Deming, 1986, 1989, 1994).
Organizational behavior can affects the quality o f services, products, and, in the case of
schools, the quality o f instruction (Deming, 1994; Patterson et al, 1986).
What has emerged from the Deming system o f profound knowledge is an evolving
body o f knowledge that incorporates systems theory, scientific method, management by fact,
and participation o f everyone within the system. Each of the quality experts mentioned have
similar messages emphasizing different concepts. Table 4 provides a matrix o f key quality
principles and the interpretation o f each offered by Deming, Juran, and Crosby.

40
Table 4. A Comparison Among Teaching Theories o f Quality Experts.
Concept Deming Juran Crosby
Definition Predictable degree of Fitness for use Conformance
o f quality dependability suitable to requirements
to market
Performance Use o f statistics to Avoidance of Zero defects
Standards measure performance campaigns to do
in all areas perfect work
Approach to Optimization o f system; Management must Prevention;
Improvement elimination o f goals consider human process
without methods side of quality development
Statistical Use SPC for Use could lead Rejection of
Process quality control to “tool-driven” statistically accepta
Control approach levels of quality
Employee Employee Use of teams; Quality
Participation participation in quality circles improvement
decision-making teams
Continuous Improvement in Business
The history o f the recent movement to improve performance in the private sector is
relevant to current and future applications in other settings. The origins o f the quality

41
movement can be traced to the 1940s and the context o f World War II (Pines, 1990). The
United States War Department established a Quality Control section in 1942 as a response to
an increased demand for mass production of weapons and other war materials. Staff from
Bell Telephone Laboratories were used— primarily two statisticians, Walter A. Shewhart and
W. Edwards Deming. Their approach was to predict the performance o f production by
measuring manufacturing processes and stabilizing their performance. When these statistical
techniques were applied, America’s defense was exemplary. At that time, the progressive
approach to manufacturing was referred to as acceptable quality levels (AQL), which
assumed that there was an acceptable level of allowable failures. The approach offered by
Shewhart and Deming suggested that this approach was one of the reasons why the United
States was seeing a decline in productivity compared to other countries.
Garvin ( I98S) reports four major quality eras. Prior to and during the 1930s, the
emphasis was on inspection. Processes for detecting defects such as grading, counting, and
repairing were common in American businesses. From the 1930s to the 1950s, statistical
quality control became popular. This strategy assumed that the principles of probability and
statistics would allow managers to control the variation in a production process to determine
if the cause o f the variation was inherent in the process or the result o f a special cause.
During the 1950s and 1960s, the quality assurance movement emphasized the planning
function, and the concept o f continuous process improvement was originated. The linkage
between quality and controlling costs was made. Beginning in the 1980s, the quality
management period was significantly influenced by W. Edwards Deming. An NBC-TV
documentary that aired on June 24, 1980, IfJapan Can, Why Can't We? explored how
Japanese products came to be perceived as far superior to those of the United States (Walton,

42
1990). During an interview, Deming, age 79 at that time, shared how he taught Japanese
management and engineers how to use quality as a system. These techniques enabled them
to detect and eliminate defects, cut down on waste, reduce costs, and increase productivity.
He used methods referred to as statistical process control (SPC). Although they had been
used in America after World War II, their use faded when volume overruled quality.
Since 1980, many companies have adopted quality principles and practices. Curt
Reimann, recently retired Director o f the Malcolm Baldrige National Quality Award,
reported in a telephone interview that all o f the high performing companies today are, in one
way or another, applying quality principles and practices. He further related that there were
many examples o f failed attempts, but companies that have successfully applied these
principles and became learning organizations are realizing results. Brown (1994) found that
some executives felt quality peaked in 1992 and many companies have abandoned quality to
resume a back-to-basics approach emphasizing results. Reimann pointed out in the interview
that the only reason to attend to processes was to improve results. This unfortunate but
common misunderstanding in the application o f quality has been substantiated by the
literature (Brown, 1994).
To help encourage United States companies and reward them for providing high
quality products and services, the Malcolm Baldrige National Quality Award was created in
1987 under President Reagan. The award was named after Malcolm Baldrige, the Secretary
of Commerce credited for his managerial approach to long-term improvement in economy,
efficiency, and effectiveness in government (National Institute of Standards and Technology,
1994). By enacting the Stevenson-Wydler Technology Innovation Act o f 1980, Congress
established the Baldrige Award which created a public-private partnership designed to

Thesis

Recommended

Recommended

More Related Content

Similar to Thesis

Similar to Thesis (20)

More from azlinazlan

More from azlinazlan (9)

Thesis