SlideShare a Scribd company logo
1 of 135
Download to read offline
THE INDIGENIZATION OF EVALUATION:
DEVELOPING A TEACHER EVALUATION TOOL FOR LITTLE WOUND SCHOOL
by
Alexander John Mackey
A School / Community Action Project
submitted in partial fulfillment of the requirements
for the degree of
Master of Arts
(Educational Administration)
in the
Graduate Studies Department
Oglala Lakota College
2016
Committee Members
Lisa White Bull, committee chair
Dr. Geraldine Gutwein
Phinette Little Whiteman
George Apple
ii
Master’s Committee
The members of the committee appointed to examine the school / community action project and
the accompanying master’s thesis of Alexander John Mackey find it satisfactory and recommend
it to be accepted.
______________________________________________
Lisa White Bull, committee chair
______________________________________________
Dr. Geraldine Gutwein
______________________________________________
Phinette Little Whiteman
______________________________________________
George Apple
iii
iv
Copyright 2016 by Alexander John Mackey. All Rights Reserved.
v
Yupćećela Oyake
Wauŋspewića-kiya pi kta ća uŋ Charlotte Danielson ća egle pi wowapi ća 1996 hehan oṫokaheya
wowapi ki le kaġa pi. Woiyuŋge wowapi waŋ etaŋ owayawa el oiṫaŋcaŋ yaŋkapi na
wauŋspewica-kiye na wayawa, iya waŋiyetu ake-śagloġaŋ sam iya pi ća hena tukte woecuŋ
iyotaŋhaŋḣći wauŋspewica-kiye wopika ća yua’taŋiŋ kta ća kaḣniga pi. Le wowapi suta ki
woiyuŋspewica-kiye wowaśi ecuŋpi ki iwaŋwicayaŋka pi kte ki uŋ ća le kaġa pi. Ṫaopi cik’ala
owayawa ki wauŋspewića-kiye taŋyaŋ ecuŋpi iwaŋwićayaŋka pi wowapi waŋ kaḣniga pi.
vi
Abstract
The Charlotte Danielson Framework for Teaching, first published in 1996, sought to create a
uniform understanding of the characteristics that make an effective teacher by categorizing
teacher behaviors into four domains of practice. This effort was in response to a fractured history
of teacher evaluation systems in the United States. In 2015, the State of South Dakota published
its requirements for instructional supervisors’ evaluations, based on the Framework for
Teaching. Schools, in beginning the implementation process, are required to choose one
component from within the Framework’s four domains. This research project was designed to
determine which components should be utilized at Little Wound School, located on the Pine
Ridge Indian Reservation. A survey was distributed to administrators, teachers, and students over
eighteen years old to determine which components are the most reflective of an effective teacher.
The data collected was analyzed using statistical tests to determine the existence of discernable
predilections among respondents. This was confirmed. Across domains one, two, three, and four,
components B, A, C, and F, respectively, proved to be preferred choices. Taking this information
into account, a model teacher evaluation tool is proposed for Little Wound School.
vii
Acknowledgements
This thesis would not be completed without the encouragement and support of those around me,
especially Mr. Russell Childree and Mr. Jesus Fuentes. Similarly, I thank my advisory
committee. Ms. Lisa White Bull’s support as the chair was a welcome foundation to my work,
and her kind words and guiding suggestions have shaped this paper greatly. Similarly, I am
thankful for Dr. Geraldine Gutwein and her discerning eyes, which have crafted this paper into a
readable and presentable work while Ms. Phinette Little Whiteman and Mr. George Apple have
influenced the orientation toward the Lakota Way. I thank my family for their work in preparing
me as a writer. And I thank Ms. Taylor Christensen for spotting an errant “dana” that was meant
to be “data.” This paper would not be the same without Mr. Jon Wenger—whose mathematical
proclivities were central to the analysis of collected data—and the time that he spent teaching an
English teacher to use SyStat and make sense of its output; this will be eternally appreciated.
Finally, I thank all the students in my classes throughout this past academic year who have heard
me talk about my time at Oglala Lakota College and my effort in writing this thesis. Regardless
of if they wanted to hear about this school community action project, they did, and the
confessional of my classroom was gallantly staffed by them.
viii
TABLE OF CONTENTS
ABSTRACT (Lakota) ………………………………………………………… v
ABSTRACT (English) ………………………………………………………… vi
ACKNOWLEDGEMENTS ………………………………………………………… vii
LIST OF TABLES ………………………………………………………… x
LIST OF FIGURES ………………………………………………………… xi
LIST OF APPENDICES ………………………………………………………… xii
CHAPTER
1. INTRODUCTION ………………………………………………… 1
Statement of Problem
Importance of Study
Definition of Terms
Limitations
Delimitations
Assumptions
2. HISTORICAL CONTEXTUALIZATION OF EVALUATION ……… 16
A History of Teacher Evaluation in the U.S.
Contemporary Research and Findings
The Implementation of Evaluation
3. METHODOLOGY …………………………………………………… 43
Subjects
Procedures and Data Collection
Data Analysis
Summary
4. FINDINGS ………………………………………………………… 61
Response Rate
Demographic Data
Findings
Statistical Symbols
5. DISCUSSION ………………………………………………………… 72
Summary
Discussion
Conclusions
Recommendations
ix
6. IMPLEMENTATION ………………………………………………… 83
BIBLIOGRAPHY ………………………………………………………… 89
APPENDICES ………………………………………………………… 107
x
LIST OF TABLES
Table 1. Rate of survey responses ………………………… 44
Table 2. Responses for Domain 1 across groups (n = 88) … 50
Table 3. Responses for Domain 2 across groups (n = 88) … 53
Table 4. Responses for Domain 3 across groups (n = 88) … 55
Table 5. Responses for Domain 4 across groups (n = 88) … 58
Table 6. Rate of survey responses ………………………… 61
Table 7. Demographics of survey respondents ………… 62
Table 8. Statistical symbols ………………………………… 71
Table 9. Variance among student, teacher, and
administrator responses ………………………… 76
Table 10. Percentage of responses for top
component preferences ………………………… 79
Table 11. Recommended Danielson Framework components
for effective teacher evaluation ………………… 81
xi
LIST OF FIGURES
Figure 1. Danielson Framework for Teaching
in comparison to the NBPTS’s Core
Propositions ………………………………………… 26
Figure 2. Distribution of Domain 1 component preferences
in relation to years of teaching experience … 52
Figure 3. Distribution of Domain 2 component preferences
in relation to years of teaching experience … 54
Figure 4. Distribution of Domain 3 component preferences
in relation to years of teaching experience … 57
Figure 5. Distribution of Domain 4 component preferences
in relation to years of teaching experience … 59
Figure 6. Evaluation tool for Little Wound School ………… 83
xii
LIST OF APPENDICES
Appendix A. Charlotte Danielson Framework for Teaching … 107
Appendix B. South Dakota S. B. No. 24: An Act ………………… 110
Appendix C. Determining Teacher Effectiveness in South
Dakota ………………………………………… 111
Appendix D. Calculating Teacher Effectiveness ………………… 112
Appendix E. Key Excerpts from the South Dakota Teacher
Effectiveness Handbook ………………………… 114
Appendix F. Teacher Effectiveness: State Requirements Checklist 115
Appendix G. Tripod Survey ………………………………… 117
Appendix H. Survey Provided to Little Wound School
Administrators, Teachers, and Students ………… 119
Appendix I. Statistical Symbols ………………………………… 123
CHAPTER 1: INTRODUCTION
Melody Schopp, Secretary of the South Dakota Department of Education, expressed her
frustration with the patchwork of teacher evaluation models profligate across the state to the
legislature. During a public hearing in Pierre, the state’s capital, on the department’s plan to
devise a new framework for evaluation, Schopp proclaimed that “we just never had clear
expectations for what’s expected of teachers” (Gahagan, 2010). But it was not enough to merely
develop a framework for the sake of having one. Schopp continued that “what I don’t want to
have happen is, ‘We’ve adopted it, and it’s done.’ We really want to move into an era of good
collaborative feedback and ‘How can I grow as a professional?’” (Gahagan, 2010).
The reasons behind her support for developing a new teacher evaluation model became
more evident as the hearing continued when she mentioned that more than ninety-five per cent of
collected evaluations ranked the observed teacher as satisfactory or above (Guilfoyle, 2013). The
panel concluded that any collected information from existing evaluations was essentially
meaningless. As a result, South Dakota realized the need to devise a new model for statewide
observation and evaluation; the state would also need to “take away the mystique of evaluations”
and provide teachers with a clearer understanding of how their own professional responsibilities
can be improved with the aid of their principals along with a more comprehensive system of
evaluation and feedback from trusted and trained supervisors (Gahagan, 2010). Any developed
framework needed to meet this primary goal: to provide a clear view of how to become a more
efficacious teacher in the classroom, therefore improving classroom instruction itself.
After the public hearing, Schopp said “you have to start somewhere” (Gahagan, 2010).
During the eighty-fifth session of the South Dakota legislature both the House of
Representatives and the Senate took up this charge and passed what came to be known as S. B.
2
24 (2010), “An act to establish standards for teaching, to require teacher evaluations, and to
provide for the development of a model evaluation instrument.” The bill did not outline what the
adopted framework for teaching should include, but instead established the mandate that the state
board of education would need to “promulgate rules... to establish minimum professional
performance standards for certified teachers in South Dakota public schools, and to establish best
practices for the evaluation of the performance of certified teachers that may be used by
individual school districts” (S. B. No. 24, 2010). (See Appendix A for S. B. No. 24’s full text.)
The state board’s working group, composed of administrators, teachers, and community
members, created an evaluation framework that ranked teachers as “distinguished, proficient,
basic, or unsatisfactory, using equal parts qualitative and quantitative measures” with the end
goal being to “make teacher evaluation more meaningful, giving school leaders better
information about which teachers deserve to be rewarded, dismissed or coached up” (Verges,
2012). In an update to the annual meeting of the Associated School Boards and School
Administrators joint convention in Sioux Falls, held August 9, 2012, the dean of the education
school at the University of South Dakota, a co-chair of the framework development working
group, expressed the importance of feedback in the state’s new evaluation model, saying that
teachers need to become active participants in their own evaluations. “Sitting down with a
teacher and saying, ‘Tell me how you know your students are learning,’ I don’t think is a bad
question to ask,” Melmer said. “We think it’s time for teachers to be more reflective” (Verges,
2012). What makes observation work is the conversation that naturally follows, the panel argued,
not the act of observation.
These ideas and others were published in April, 2015, as the South Dakota Teacher
Effectiveness Handbook: Requirements, Support Systems, and State Model Recommendations,
3
the consequence of four years of internal development, community input, and work with local
education agencies—including schools, districts, and leaders—across the state. Published by the
South Dakota Commission on Teaching and Learning, this document outlined the state’s
practical guidelines for implementing the group’s supervisory evaluation program.
The state decided to base its observation model on an established guideline first
organized and published in 1996 by Charlotte Danielson, who developed her framework for
teaching while working for Educational Testing Service (ETS), the world’s largest maker and
administrator of standardized assessments (Mercer, 2013; Singer, 2013).
The Charlotte Danielson Framework for Teaching. As it has come to be known, the
Danielson Framework (throughout this writing, the phrase “Danielson Framework” is used
interchangeably with variations including “the Danielson Model,” the “Framework for
Teaching,” or “Framework for Effective Teaching,” for instance) was chosen to serve as the state
model for observation, evaluation, and feedback. Outlined in Enhancing Professional Practice:
A Framework for Teaching, the book first published in 1996 by Charlotte Danielson, her
framework is divided into four primary domains—these include planning and preparation, the
classroom environment, instruction, and professional responsibilities—each comprised of
between five and six components, which serve to add depth and provide a more concreate
understanding of what the four domains include. The components are broken down into seventy-
two elements that further elaborate on the components. For example, within the first domain,
“planning and preparation,” the first component is 1a: “demonstrating knowledge of content and
pedagogy,” which itself is sub-divided into three elements: “knowledge of content and the
structure of the discipline,” “knowledge of prerequisite relationships,” and “knowledge of
4
content-related pedagogy” (Danielson, 2007). (For a listing of the Danielson Framework’s
entirety, see Appendix A.)
Danielson postulates that a “framework for professional practice” is important “because
teaching is complex, [and therefore] it is helpful to have a road map through the territory,
structured around a shared understanding of teaching” (p. 2). The Danielson Model is important,
its developer argues, because “a framework for teaching offers educators a means of
communicating about excellence.” As such, when “conversations are organized around a
common framework, teachers are able to learn from one another” and the framework’s domains,
components, and elements “are validated for any particular setting” (p. 6).
South Dakota has adopted the Danielson Framework under its own moniker: the South
Dakota Framework for Teaching, which is an exact reproduction of Danielson’s work (South
Dakota Commission on Teaching and Learning, 2015, p. 10). Districts within the state are not
required to utilize this particular measurement framework, but any district seeking an exemption
must provide their own model that is directly aligned to the South Dakota Framework for
Teaching. The State of South Dakota further notes that while the framework “includes twenty-
two individual teaching components clustered into four domains,” district “effectiveness systems
must include professional performance evaluations based on a minimum of four teaching
components, including one from each domain” (p. 10). The Commission on Teaching and
Learning, however, recommends that eight components be used as the basis of teacher
observation and evaluation; it is the purview of the local education agency to determine which
components are the most beneficial to the development of strong teachers (p. 16).
Based on these domains and their components, and being coupled with student learning
outcomes on state, standardized, and district assessments, the framework is designed to provide a
5
particular evaluation outcome: principals derive a Summative Teaching Effectiveness Rating that
leads to a teacher being identified as below expectations, meeting expectations, or exceeding
expectations. This summative teacher rating “includes an evaluation of student growth that
serves as one significant factor and an evaluation of professional practices as the other significant
factor” (p. 12). (See Appendix C for an outline for how effectiveness is measured; see Appendix
D for a model provided by the state on how to calculate a teacher’s effectiveness score.)
According to a timeline for implementation, the as-designed South Dakota Framework
for Teaching will be fully implemented in public schools in the state for the 2015–2016 school
year (South Dakota Commission on Teaching and Learning, 2015, Appendix B).
Statement of Problem
This research project seeks to identify what those elements of instruction are in a single
school located on the Pine Ridge Indian Reservation, which is in southwestern South Dakota and
home to the Oglala Sioux Tribe: Little Wound School. As the State of South Dakota and Bureau
of Indian Education begin to require that schools adopt a teacher evaluation tool based on the
Charlotte Danielson Framework for Teaching, it is necessary that local education agencies
identify aspects of the framework most pertinent to teaching and learning in a classroom setting at
each unique school site. Furthermore it is especially true for a school that serves primarily Native
American students.
Because schools have the leeway to decide how many and which components of the
Danielson Framework will be evaluated (so long as it meets the minimum requirements of
observation outlined by the South Dakota Department of Education) by supervisors, it is
important that decisions made in this process be well informed and data-driven. In a search of
literature, no published, peer-reviewed journal articles expressly sought to apply the Danielson
6
Framework to a tribally operated school, but existing research has shown time and again that
students of diverse demographic backgrounds have inimitable educational needs (Koon,
Petscher, & Foorman, 2014).
This school-community action project sets out to determine which components of the
Danielson Framework make the most effective teacher, as identified by students, teachers, and
administrators at Little Wound School. With this information in hand, results can be reviewed,
trends tracked, and proposals for practice proffered. Up to this point, data has not been collected
from the Little Wound School District that answers this question. Through this collection
process, answers to the following questions will be sought:
1. Will students, teachers, and administrators have different or similar views about the
components of the Danielson Framework that make for a most effective teacher?
2. Can variations in the data collected from teachers and administrators be attributed to
differences in the length of teaching experience, gender, position of employment, or
other identifiable factors?
3. Will the results from students mirror or juxtapose those collected from teachers and
administrative staff?
4. Are there particular components of the Danielson Framework that are unanimously
believed by those individuals surveyed to be the most important factor in identifying
a most effective teacher?
It is the belief of the researcher that within each of the four domains of the Danielson
Framework, one particular component will stand out as the most often cited factor in making for
a most effective teacher. The information collected from this process will be used to guide the
creation of a proposed evaluation tool for supervisors and principals to use as they observe
7
teachers and classrooms for instructional effectiveness. This proposed evaluation tool will reflect
the beliefs and views of teachers throughout all grade levels of the school, principals and other
administrators, as well as a sample of adult-aged students.
Importance of Study
This school-community action project seeks to indigenize the Danielson Framework for
application at Little Wound School. By explicitly working to identify which components of the
framework are viewed by students and staff as the most applicable in determining teacher
effectiveness in this one particular setting, the Danielson Framework shifts from being an
imposed method of evaluation to becoming a locally influenced tool for gauging teacher
effectiveness. This process allows Little Wound School to utilize its own people’s knowledge in
discerning this important measure.
Schools that operate within the province of the Bureau of Indian Education (B.I.E.), of
which Little Wound School is included, have for the past seven years been encouraged to utilize
the Danielson Framework to evaluate teacher effectiveness. But the B.I.E.’s requirement to
utilize the Danielson Model comes within a larger group of recommendations as part of the
NativeStar regimen of “rapid improvement indicators” and “transformation implementation
indicators” (Bureau of Indian Education, 2015). These different indicator groups (there are
ninety-nine individual indicators for school improvement in the former group and twenty-four in
the latter) only allude to the importance of utilizing the Danielson Framework as a foundation for
classroom and instructional improvement. Each of these indicators are termed a “WiseWay.”
For example, in WiseWay indicator forty one—“Indicator: The principal maintains a file
of the agendas, work products, and minutes of all [instructional] teams”—schools are specifically
directed to observe Danielson’s encouragement that “teacher evaluation should serve as an
8
opportunity for professional development,” (Academic Development Institute, 2013a). Likewise,
in WiseWay indicator one-hundred-four—“Indicator: Yearly learning goals are set for the school
by the Leadership Team utilizing student learning data”—Charlotte Danielson’s seminal book,
Framework for Teaching, is referenced as being an important tool in being able to both
understand and evaluate “multiple methods of assessment to engage learners in their own
growth, to monitor learner progress, and to guide the educator’s and learner’s decision making.”
(Academic Development Institute, 2013b). In totality, out of the collection of indicators, the
works and ideas of Charlotte Danielson are directly referenced ten times. Consequently, in order
for a Bureau of Indian Education grant school to fully adopt the NativeStar improvement
indicators, it is central that schools adopt the Danielson Framework as a matrix by which to
evaluate teachers and learning.
All schools that operate within the Bureau of Indian Education are expected to adopt a
NativeStar-based school improvement mindset. Once this is established, it is up to internal
school policy-makers to determine how this is done—the Bureau does not provide additional
direction. Schools are encouraged to look at their home state’s recommendations.
For Little Wound School, this is South Dakota. More specifically, it is the state’s report,
the South Dakota Teacher Effectiveness Handbook: Requirements, Support Systems, and State
Model Recommendations. This monograph additionally requires schools utilize the Danielson
Framework (or the “South Dakota Framework for Teaching,” whose content is the same) for
teacher evaluation. Consequently, Little Wound School—on both the state and federal side—is
being prodded to adopt this framework.
Little Wound, in the first years of implementation, need only evaluate teachers on at least
one component in each domain—four total. This school has yet to establish a standard to be used
9
across all schools (elementary, middle, and high) by all supervising administrators. While the
school may wish to seek established recommendations about which components to choose, no
existing research has been conducted that indicates which components are most efficacious to
determine this information; similarly, there have been no internal research activities or surveys to
determine what currently employed staff members and enrolled students believe to be the
components that elucidate this information.
The development of a teaching evaluation framework for Little Wound School is
important because not only is it a requirement of the State of South Dakota and the Bureau of
Indian Education, but little research has been conducted about what elements of teaching are the
most important in determining effectiveness in a Native American school setting, especially
when aligned to the Danielson Framework, which this research seeks to do.
While some research indicates ways that Native American students generally learn best in
a classroom setting (Tharp & Yamauchi, 1994; Morgan, 2009), no discernable research has been
conducted about what Native American educators, students, and administrators believe are the
traits that lead to effective teaching, especially when considered on the Pine Ridge Indian
Reservation, the home of Little Wound School. This research project aims to unveil some of
these individual perceptions and yield a teacher evaluation framework that is indigenous to Little
Wound School and its students and staff.
Definition of Terms
In order that this thesis be understood in its entirety, it is important to define some key
terms used throughout the paper. These definitions are curated from selected publications and
reviews and articles that provide the most complete sense of the word’s true contextual meaning.
10
Achievement and Learning: Student achievement is “the status of subject-matter
knowledge, understandings, and skills at one point in time” while student learning is “growth in
subject-matter knowledge, understandings, and skill over time. In essence, a change in
achievement constitutes learning. It is student learning—not student achievement—that is most
relevant to defining and assessing accomplished teaching” (Linn, Bond, Carr, Darling-
Hammond, Harris, Hess, & Shulman, n.d., p. 28).
Clinical supervision: A process of supervision with three features: (1) autonomy: “the
goal is for the teacher to become more self-directed and analytical,” (2) evidence: “the evidence
for change in behavior arises from the observational data,” and (3) continuity: “the process
unfolds over time” (Kilbourn, 1982).
Domain: A particular “sphere of activity or knowledge” (“domain,” 2015). Within proper
context, a domain is also one of four areas of supervision as identified in the Danielson
Framework, which include: planning and preparation, classroom environment, instruction, and
professional responsibilities (Danielson, 2007).
Effective: This term is used to describe a teacher who has been “the most successful in
helping [students] to learn” information and content (Walker, 2008, p. 63).
Evaluation: A process to determine various aspects of teaching (a teacher’s effectiveness)
that includes three essential elements: (1) “a coherent definition of the domain of teaching (the
‘what?’), including decisions concerning the standard for acceptable performance,” (2) analysis of
“techniques and procedures for assessing all aspects of teaching (the ‘how?’),” and (3) the process
is done by “trained evaluators who can make consistent judgments about performance, based on
evidence of the teaching as manifested in the procedures” (Danielson & McGreal, 2000).
11
Improvement: A progressively unfolding process whereby achievement in a particular
subject or area gets better in a measurable manner; it does not “have a fixed or predetermined
end point, and that is sustained over extended periods of time” (Hidden Curriculum, 2014).
Instruction: From the Association for Supervision and Curriculum Development, a
definition of instruction is provided: “Instruction consists of any steps taken in planning and
conducting programs of studies and lessons that suit them to the… students’ learning needs,
learning readiness, and learner characteristics” (Heathers, 1977, p. 342)
Model / framework: Used interchangeably, broadly, it is a “system of concepts,
assumptions, expectations, beliefs, and theories that supports and informs your research” and
practice (Miles & Huberman, 1994). Specifically, in the field of education, within a context of
supervision and evaluation, it is a “blueprint for implementation” of a particular theory or
practice to improve classroom instruction (California Department of Education, 2015).
Observation: As provided by the South Dakota Commission on Teaching and Learning,
there are two distinct types of observations: formal observations, which are “scheduled
observation[s] of teaching practice conducted by an evaluator that is at least 15 minutes in length
and includes structured conversations before and after the observation takes place” and informal
observations, which is “an observation of teaching practice, which may or may not be
announced, that is conducted by an evaluator, and is at least five minutes in length, and results in
feedback to the teacher” (2015, p. 38).
Professional development: This term means a “comprehensive, sustained, and intensive
approach to improving teachers’ and principals’ effectiveness in raising student achievement”
that “is aligned with rigorous state [and local] student academic achievement standards,” “is
conducted among learning teams of educators, including teachers, paraprofessionals, and other
12
instructional staff at the school,” and finally “engages established learning teams of educators in
a continuous cycle of improvement” (National Staff Development Council, n.d.).
Value-added model: An assessment that analyzes “test data that can measure teaching
and learning” which is based on “a review of students’ test score gains from previous grades” in
order to determine whether students have “made the expected amount of progress, have made
less progress than expected, or have been stretched beyond what they could reasonably be
expected to achieve” (University of Pennsylvania, 2004).
Limitations
Limitations within this school-community action project are beyond the control of the
researcher but may impact the study’s overall worth. Although this research was carefully
planned, organized, analyzed, and presented, some of the limitations that affect this particular
thesis include the following:
First, not all teachers have had professional training on the Charlotte Danielson
Framework and, consequently, do not have a full understanding of the material surveyed. In the
spring of 2013, Little Wound School hosted a full-teaching staff professional development with a
representative from the Danielson Group and all teachers were given a copy of Enhancing
Professional Practice: A Framework for Teaching. No teachers have been formally trained on
the framework in the three subsequent years and no further books have been given out. Teachers
who became employed by Little Wound School in subsequent years may not have ever been
exposed to the Danielson Group’s Framework for Teaching.
Also, the population surveyed was narrow—staff, administration, and some students at
Little Wound School during the 2015/2016 school year—and therefore cannot be immediately
generalized beyond this one particular local education agency. Results or suggestions for
13
implementation may not be generalizable, even to schools that share similar demographics or
environments.
Not all students at the school were surveyed—only students over the age of eighteen shall
be participants in this research. This eliminates the study’s work with a vulnerable population
(below eighteen years of age) while simultaneously permitting the researcher to provide
identically worded survey to all participants and, subsequently, removing concerns about the
reading level of the material provided. This group will still provide valuable information.
It must also be noted that the data collected is a snapshot of one particular point in time.
Each individual who completed the survey will only take the survey once. Consequently,
circumstances within the school environment may change and, as a result, impact people’s
perceptions of supervisory observation and evaluation. For example, a change in school
leadership or within the school board could see a reevaluation of teacher evaluation, new focuses
in what to evaluate, and new metrics by which to gauge how effective individual teachers are.
Similarly, as teachers come and go, overall staff perceptions may shift or change, endangering
the long-term usability of this information.
Finally, any identified correlation does not equal causation. Even if all the individuals
surveyed provided the same responses, there could be no direct line drawn between what people
think makes for a most effective teacher and what truly does make a teacher great.
Delimitations
Delimiting factors fall within the control of the researcher and could impact the quality of
the work collected. In pursuit of transparency, some of the most important decisions that could
produce altered results are included below. These decisions have been carefully weighed,
14
however, and were made with the intention of reducing errant outcomes and minimizing the
opportunity that research collected cannot be used.
First, only individuals employed by or attending Little Wound School at the time the
survey was conducted were eligible to be included in the research.
Second, on the survey, reference was made to the Danielson Framework for Teaching,
which is often associated with evaluation and observation, but on the survey utilized, the
question posed for the individual answering survey read: “What is the most important element in
each box that makes for the most effective teacher?” The Danielson Framework is designed to
measure excellence in teaching, which warrants the identification of the effectiveness of
particular teachers. If teachers were asked which of the components of the Danielson Framework
are most important in a principal’s observation of a classroom, the information might merely
reflect what is observable, what is base, and (potentially) what is easiest.
It should also be noted that a survey was the sole method of information collection for
this research. There is no included ethnographic element, and the methodological evaluation is
entirely quantitative. It should be noted, however, that because of the sample size, data collected
is legitimate and can be utilized to create a localized teacher evaluation system.
Finally, the content of the survey distributed utilizes the same wording and phrasing as
developed by Charlotte Danielson. The vocabulary is of a high level and some participants might
not be fully versed in the vocabulary. The decision has been made to maintain this wording
because it provides the most accurate reflection of the school’s present understanding of the
Framework as well as maintains Danielson’s original intent.
15
Assumptions
The following assumptions are made regarding the research for this writing and for the
continued use of any data collected or patterns discerned.
A first major assumption of this thesis is that Little Wound School will, per the directive
of the Bureau of Indian Education, continue to utilize the Danielson Framework as the primary
tool of classroom observation and teacher evaluation. Should policy change at the national, state,
or local level, and the Danielson Framework were to be replaced or modified, the survey and the
results collected and analyzed herein would be moot.
This survey also anticipates that answers collected are based on truthful and honest
responses from the survey participants.
16
CHAPTER 2: HISTORICAL CONTEXTUALIZATION OF EVALUATION
A History of Teacher Evaluation in the United States
Evaluation of teachers in the United States can be tracked back to the days that schools
were first being established in disparate communities in the 1700s. At this time, there was no
unified professional discipline of education, and responsibility for both founding and monitoring
schools and teachers fell on the town in which it was located and upon the parents whose
children attended that particular school; the community often appointed an individual or
established a supervisory committee that set the standards by which a teacher served the town
(Burke & Krey, 2005). As a result of this lack of pedagogical consistency, the quality of teachers
could vary greatly from place to place and school to school.
This school-by-school variation began to coalesce into centralized administration in the
1800s with the “common school” movement, whose biggest proponent was Horace Mann, the
first secretary of the State Board of Education in Massachusetts (Simpson, 2004). While these
informal early school districts had a superintendent, the burden of administration typically fell
upon a ‘principal teacher,’ who, in time, become simply the ‘principal,’ as the title is understood
today (Marzano, Frontier, & Livingston, 2011).
As schools become more professionally managed, a debate began to arise in the late 19th
and early 20th
century—education’s first crisis of purpose. One of these two competing views of
education was spearheaded by John Dewey, who proposed that democracy was the true
underpinning of societal progress and that the purpose of schools can only properly be measured
by how well they encouraged civic action and participation in civil society (DeCesare, 2012). In
Dewey’s view, “Progressive ideas such as a student-centered education, connecting the
classroom to the real world, differentiation based on student learning needs, and integration of
17
content areas were espoused” by the thinker as “ways of bridging the gap between students’
passive role as learners and the active role they would need to play as citizens” (Marzano et al.,
2011).
The opposing camp was manifested by Frederick Taylor, who believed that any
organization must find the most effective manner by which to accomplish their goals, then
universalize and standardize that particular practice (Freedman, 1992). This latter philosopher’s
ideas, in the early 20th
century, came to embody how schools evaluated their success. Pushed
forward by the writings of a Harvard professor, Edward Thorndike, and applied to education by
Ellwood Cubberley, schools evaluated teachers in a same way that factories measured their own
productivity. In Cubberley’s 1916 book Public School Administration, the author states that:
Our schools are, in a sense, factories in which the raw products (children) are
to be shaped and fashioned into products to meet the various demands of life.
The specifications for manufacturing come from the demands of twentieth
century civilization and is the business of the school to build its pupils
according to the specifications laid down (p. 338).
It was from this factory-model of education that Cubberley established his understanding for
how school success should be measured, which emphasized “measurement and analysis of data
to ensure that teachers and schools were productive” (Marzano et al., 2011).
These two competing views stood in opposition to one another for decades to come,
reinforced by various philosophers, academics, and school leaders. But as time progressed,
individuals within the educational field began to see this as a false dichotomy; the views of
Taylor and Cubberley provided a feedback loop that encouraged teachers to be more effective,
leading students to a greater involvement in civil society, which was the idea that Dewey
18
espoused more than a century earlier (Raths & McAninch, 2005, p. 164). It is from here that a
school’s dual purpose, which became more pronounced in the post-World War Two era, become
evident: that schools must be data-driven institutions in pursuit of broad goals established by
society and the community (Marzano et al., 2011). It is this two-pronged approach that gave rise
to one of the most widely adopted methods of school and teacher evaluation: clinical
supervision.
Coming into the forefront of educational thought in the 1950s, clinical supervision is best
known as culminating to and emanating from a 1969 book written by Robert Goldhammer,
Clinical supervision: Special methods for the supervision of teachers; this book established the
basic need of school leadership to maintain close communication with their teachers, observing
their practice, and providing appropriate and immediate feedback for improved instruction
(Goldhammer, 1969). By 1980, surveys found that more than ninety percent of school leaders
used Goldhammer’s approach to school evaluation in their practice (Marzano et al., 2011).
A 1973 work by Morris Cogan—Clinical Supervision—further elaborated on the
intricacy of the practice, stating that:
A cornerstone of the supervisor’s work with the teacher is the assumption that
clinical supervision constitutes a continuation of the teacher’s professional
education. This does not mean that the teacher is “in training,” as is sometimes
said of preservice programs. It means that he is continuously engaged in
improving his practice, as is required of all professionals. In this sense, the
teacher involved in clinical supervision must be perceived as a practitioner
fulfilling one of the first requirements of a professional—maintaining and
developing his competence. He must not be treated as a person being rescued
19
from ineptitude, saved from incompetence, or supported in his stumblings. He
must perceive himself to be engaged in the supervisory processes as a
professional who continues his education and enlarges his competences (p. 21).
The clinical supervisory model, however, had its downfalls and detractors: it was considered
“didactic” and “formulaic,” and that “supervisory and evaluative approaches that were more
developmental and reflective were sometimes viewed as not specific enough to enhance
pedagogical development” (Wise, Darling-Hammond, McLaughlin, & Bernstein, 1984). It was
believed among surveyed teachers that there were four primary issues with the clinical
observation model of teacher evaluation: (1) principals did not have the “resolve and
competence” to accurately evaluate teachers; (2) teachers, overtly or covertly, resented receiving
critical feedback; (3) there was no uniform evaluation measures by which standards of
instruction could be applied to all teachers evenly; and (4) was that little or no training was
available for principal-evaluators, which led to inconsistency in implementation (Wise et al.,
1984, p. 22).
From these critiques rose a seminal work, written by Charlotte Danielson in 1984:
Enhancing Professional Practice: A Framework for Teaching. Danielson’s background was
working for Educational Testing Services (ETS), and in particular working on identifying
measurable ways by which to effectively “measure the competence of teachers” (Marzano et al.,
2011). Historical models of evaluation focused on singular aspects of the teacher’s actions, be it
the steps of the teaching process or the supervisory process, for instance, but Danielson sought to
create a comprehensive model for supervisory observation that captured the dynamic processes
involved in classroom teaching.
20
The Danielson Framework, as it has come to be known, is composed of four domains:
Planning and Preparation, the Classroom Environment, Instruction, and finally Professional
Responsibilities; each of these domains is then broken down into subdomains, seventy six in all,
which seek to work in tandem and provide a common language of evaluation for all stages in the
teaching process, from planning to reporting of data (Danielson, 1984).
The twenty-first century saw a modification of Danielson’s Framework in an effort to
incorporate a more student-centered view. This new paradigm was championed by Tucker and
Stronge and outlined in their 2005 book Linking Teacher Evaluation and Student Achievement,
which advocated the “importance of student achievement as a criterion in the evaluation process”
and “argued for evaluation systems that determine teacher effectiveness using evidence from
student gains in learning as well as observations of classroom instruction” (Marzano et al.,
2011). It is the work of these two individuals that is accredited with the popular rise of value-
added (quantitatively-based) teacher evaluations, similar to the ones adopted and published by
the Los Angeles Times in 2010 and 2011. The authors forcefully state in their recommendations
that, “given the clear and undeniable link that exists between teacher effectiveness and student
learning,” this should “support the use of student achievement information in teacher assessment.
Student achievement can, and indeed should be, an important source of feedback on the
effectiveness of schools, administrators, and teachers” (p. 102).
In the last decade, many of the models of teacher evaluation have come under siege and
are chided as “superficial, capricious, and often don’t even directly address the quality of
instruction, much less measure students’ learning” (Toch & Rothman, 2008, p. 1). Toch and
Rothman, in their 2008 study Rush to Judgement, found that, in spite of No Child Left Behind’s
requirements for more rigorous forms of teacher evaluation, only fourteen states, at the time,
21
even required a single observation of a principal for a teacher; many of these evaluations only
required supervisors to check boxes that teachers were “satisfactory” or “unsatisfactory” (Toch
& Rothman, 2008). Additional contemporary reports found similarly troubling findings that
demonstrated that a rigorous form of teacher observation and evaluation was still nothing more
than an ethereal end-point. Research that took the authors to twelve school districts across four
states—data was collected including on more than fifteen thousand teachers, one thousand three
hundred administrators, and nearly one hundred district-level administrators—identified trends
that were troubling portents illustrating deep flaws in teacher evaluations in major American
school districts (Weisberg, Sexton, Mulhern, & Keeling, 2009). These researchers found that
school districts tend to assume that classroom effectiveness is:
the same from teacher to teacher. This decades-old fallacy fosters an
environment in which teachers cease to be understood as individual
professionals, but rather as interchangeable parts. In its denial of individual
strengths and weaknesses, it is deeply disrespectful to teachers; in its
indifference to instructional effectiveness, it gambles with the lives of
students (p. 4).
Weisberg, Sexton, Mulhern, and Keeling, the writers of the report (who were assisted by
contributing authors including Schunck, Palcisco, and Morgan) propose that the history of
teacher evaluations is fraught with failures of understanding; inconsistent application of
evaluations by principals, schools, districts, and states; a false understanding of the differences in
instructional styles of teachers; and too short and too infrequent to be beneficial. All of these
results, in the end, create a system where “excellent teachers cannot be recognized or rewarded,
chronically low-performing teachers languish, and the wide majority of teachers performing at
22
moderate levels do not get the differentiated support and development they need to improve as
professionals” (p. 6). It is clear, the report concludes, that the current system of evaluation is not
a practical or useful model and must be abandoned for something more useful and responsive.
Contemporary Research and Findings
In order to gain a more thorough understanding of contemporary forms of observation
and evaluation, it is necessary to abandon a general historiographical approach in favor of a more
sharpened approach—it is necessary to analyze recent research reports that look at what, in fact,
these types of evaluative and supportive relationships between principal and teacher look like in
practice and in the field of education itself. Consequently, in the pursuit of a modern view of
educational observation, it is central that only research from the twenty-first century be
incorporated into a review of literature on this particular topic.
Before exploring how to measure an effective teacher, though, it is important to look at
how an effective teacher is being defined. In a 2008 report by Robert Walker, entitled Twelve
Characteristics of an Effective Teacher: A Longitudinal, Qualitative, Quasi-Research Study of
In-Service and Pre-Service Teachers’ Opinions, the loaded term “effectiveness” in education is
defined as a “particular teacher who had been the most successful in helping respondents to
learn” (p. 63). While broad in scope, it permits Walker to discern twelve characteristics that are
generally associated with the most effective classroom instructors: being prepared; maintaining a
positive attitude; holding students to high expectations; encouraging creativity; treating students
equitably and fairly; displaying a “personable, approachable touch with students”; cultivating a
sense of belonging among students; dealing with students compassionately; the possession of a
sense of humor; demonstrating respect; being forgiving; and the admission of mistakes (p. 64).
23
Beginning in the year 2002, a not-for-profit organization, Teach For America, which
takes primarily recent college graduates and, with one month of training, places these individuals
into low-income, high-need communities, began to study its internal data in the pursuit of what
makes for phenomenal teachers, whose students often grow by more than one and a half years
within the timeframe of a single academic year (Ripley, 2010). This research was led by a chief
researcher within the organization, Stephen Farr, who published a book with many of his
findings in 2010: Teaching as Leadership: The Highly Effective Teacher’s Guide to Closing the
Achievement Gap. In the book, Farr boldly claims that his primary finding is that “strong
teachers insist that effective teaching is neither mysterious nor magical. It is neither a function of
dynamic personality nor dramatic performance” but instead can be clearly delineated in a style
that can be applied to many teachers (Farr, 2010). (Farr is careful to qualify his findings in that
this is not the only way to achieve effectiveness in classroom instruction.) In his book, Farr
outlines concrete and actionable examples that were common traits in high-level classrooms.
These findings include the presence of “big goals” that drive classroom teaching and learning,
consistent and meaningful checks of understanding, building investment with students and their
families, purposeful planning, and continually seeking personal and professional growth, among
others (Farr, 2010, p. vii–ix). Many of these findings were independently confirmed by a
Mathimatica Policy Research report, which not only found similar common traits among
effective teachers, but also that students in classrooms with Teach For America trained and
supported teachers, “significantly outperformed those [classrooms led by] their more
experienced counterparts” (Decker, Mayer, & Glazerman, 2004; Ripley, 2010).
One of the selling points of Teach For America for prospective school districts seeking to
hire its participants, or “corps members,” is that the organization provides a significant amount
24
of classroom support (including observation, evaluation, suggestions, and professional
development) for teachers that supplements what schools and districts seek to do with their
principal’s supervisory responsibilities (Cody, 2012). Teach for America seeks to employ
multiple measures of student and teacher growth to develop a holistic, but nonetheless data-
backed, system of measurement to determine effectiveness (Sawchuk, 2009).
This multifaceted approach to teacher observation and evaluation has been practiced in
Pittsburgh for an extended period of time and is the subject of a research paper, published in
2014. The researchers look at how Pittsburgh Public Schools utilizes a three-pronged approach to
gauge a teacher’s effectiveness. These three domains of evaluation (professional practice, student
surveys, and value-added measures) are positively correlated, which suggests, according to the
researchers, that “they are valid and complementary measures of teacher effectiveness” (Chaplin,
Gill, Thompkins, & Miller, 2014).
The first component of the observation and evaluation regimen that the Pittsburgh Public
Schools requires is a professional practice measure that is based on the Charlotte Danielson
Framework for Teaching, known as the Research-based Inclusive System of Evaluation (RISE),
which relies on principal evaluations of teachers. This is coupled with a student survey that seeks
to “incorporate students’ perceptions of teachers’ practices and was developed by Ronald
Ferguson of Harvard University as part of the Tripod Project and administered by Cambridge
Education” (Bill & Melinda Gates Foundation, 2012). The third component of evaluation is
based on a value-added measure that charts student changes on test scores over the three
previous years’ worth of teaching (Chaplin et al., 2014). This multi-domain approach allows
school leaders and administrators to develop a more well-rounded approach to teacher
observation and evaluation that could not be obtained by merely looking at any one of these three
25
areas independently. The researchers suggest, through their findings, that the comparability and
positive correlation of the three types of evaluations strengthens the fact that they should be used
to complement one another, not supplant other measures of effectiveness. “Although none of the
measures represent a gold-standard benchmark, the correlations across them suggest that they are
capturing various aspects of effective teaching in complementary ways” (Chaplin et al., 2014, p.
iii). It is the use of a modified version of the Danielson Framework that warrants continued
analysis and discussion.
Thomas Viviano, in a 2012 research paper published in the Journal of Career and
Technical Education titled “Charlotte Danielson or National Board Certification: A Comparison
and Contrasting of Two Major National Frameworks for Teaching,” seeks to differentiate
between the two major national models of teacher evaluation, identified in his quasi-eponymous
title. The Danielson Framework includes four domains of professional operation (planning and
preparation, the classroom environment, professional responsibilities, and instruction) that are
divided into twenty three sub-domains that provide additional areas for observation and
evaluation. Teachers operating in this framework are classified—within each domain or sub-
domain—as one of “four levels of competency to include distinguished, proficient, needs
improvement or progressing, and unsatisfactory” (Viviano, 2012, p. 114). The National Board
for Professional Teaching Standards (NBPTS) measures “five core propositions on what teachers
should know and be able to do” and within these five areas there “are a total of sixteen subject
matter areas that a teacher can earn a certificate in within various developmental levels for a total
of twenty five certificates in all” (p. 115). These Danielson domains are compared with their
most closely related NBPTS propositional area in the figure below, Figure 1, “looks at the
comparisons of the two national frameworks and how they cross reference.” (p. 116).
26
Figure 1
Danielson Framework for Teaching in comparison to the NBPTS’s Core Propositions
Charlotte Danielson’s
Four Domains
NBPTS’s Core
Propositions
Planning and
Preparation
Teachers are committed to
students and their learning
Classroom
Management
Teachers know the subjects
they teach and how to teach
those subjects to students
Instruction
Responsible for monitoring
and organizing student
learning
Professional
Responsibilities
Teachers think
systematically
about their practice and
learn from their experience
Teachers are members of
learning communities
This figure “looks at the comparisons of the two national frameworks and how
they cross reference. NBPTS has two categories that engulf professional
responsibilities and two for instruction” (Viviano, 2012, p. 116).
Between the Danielson Framework and the NBPTS, there are two minor differences that
Viviano identifies; these differences relate to “using data to help guide and plan curriculum and
teaching methods, and showing professionalism. Danielson’s framework addresses both of these
categories and NBPTS addresses neither” (2012, p. 118). In the national standards for teaching
excellence outlined in the legislation and policy surrounding the George W. Bush-era No Child
Left Behind Act, “data collection to improve teaching and learning has become very important
27
and is not only the responsibility of administration but also now the responsibility of teachers,”
and the presence of this measurement within the Danielson Framework lends credence to this
model’s inclusivity of contemporary issues in educational instruction and supervision (p. 118).
Apart from the non-inclusion of data in the NBPTS, he also discusses the Danielson
Framework’s incorporation of professionalism as a matter of professional practice, further stating
that it is “an essential component if you are going to use a framework for assessment and
evaluation” because it is important for educators to “look at the comprehensive teaching
professional and professionalism as crucial in that so many young lives are dependent on role
models to pave the way toward strong and responsible citizenship” (p. 118).
Viviano, in his conclusions, brings up important points that relate to how the Danielson
Framework and the NBPTS should be used in evaluations and assessments, bringing up the
important observation that any time a principal or supervisor “evaluates a teacher, he or she is
placing a worth on another human being’s skills” (p. 118). It is important, he conveys, that after
an assessment, “the administrator and teacher should concern themselves with what can be done
to ameliorate any problems or skill deficiency that was revealed during the assessment process in
order to benefit students.” It is from here, with the understanding that comes from utilizing a
professionally developed standard of evaluation such as the Danielson Framework, that the
administrator then “merely becomes the facilitator to make sure that the teacher goes through the
right professional development needed such as a workshop, mentoring from a fellow teacher, a
coach, the administrator, or research” (p. 118).
It is this intricate professional relationship shared between teachers and their supervisors
that is the subject of Callahan and Sadeghi’s research, published in 2015, that looks at teacher
perceptions of evaluation models in New Jersey’s public schools. In 2012, the state legislature of
28
the State of New Jersey—in response to motivation from the Obama administration’s Race to the
Top program—signed into law a program that “calls for a four level evaluation system of
teachers that links individual student data to teachers and creates a more difficult process for
teachers to earn tenure” (p. 47). Boldly, the law also “targets teachers who have already earned
tenure” and, “in a major change to educational policy, tenured teachers may lose their jobs after
two consecutive years of ineffective evaluations” (p. 47). In order to gain a greater understanding
about this valued-added coupled with principal evaluation approach, the researched conducted a
survey of more than six hundred eighteen public school teachers across the state. The findings of
this survey prove illuminating.
Callahan and Sadeghi identified that the state-required minimum observation
requirements did increase the quality and duration of classroom evaluations by principals: in
2012, the first year of implementation, “formal evaluations were conducted infrequently with a
varying degree of accuracy and impact” and “nearly half of the teachers indicated the formal
evaluations did not lead to improvements in their practice” (p. 56). But in a follow up survey
conducted in 2014, two academic years after the program’s start, teachers did identify an
increase in the rate of observation, but concerns were raised about the diminishing value of the
observations themselves. A formulaic observation regimen, they argued, led teachers to be more
concerned with the technical process of observation as compared to its supposed end goal:
improving classroom instruction. Several teachers “noted that their principals were more focused
on entering observations in real time then on teacher-centered observations. They appeared more
focused on entering information on tablets, then in actually observing.” Teachers further noted
that “the technology and demands of observing numerous required elements made the
observation scripted” and, therefore, unproductive (p. 56).
29
In their conclusion, the authors anchor their findings to the programmatic intentions of
the teacher evaluation and tenure system, known as ACHIEVE NJ, identifying that the process of
required minimum observation standards by principals has transformed, with detrimental effects,
what was once “an organic, albeit infrequent, process” into a scripted one. This shift has
produced generally demoralized educators, of which one contributing factor is the state-
mandated emphasis on rating teachers (p. 56). The writers’ findings are summarized in the final
paragraph of this particular piece of peer-reviewed literature, remarking that:
teacher evaluation systems are not perfect and effective teachers are not the
product of formulas. Research shows us that much of what effective teachers
do cannot be measured by categorical ratings. However, that is not to say we
should not attempt to define what effective teachers do and make every effort
to replicate it. We need to move beyond checklists and rubrics that fail to
acknowledge teaching excellence and we need to identify and offer
professional development strategies that are most effective in improving
teaching pedagogy and ultimately improving student achievement (p. 56).
But if teachers are demoralized from being categorically ranked and evaluated, then not provided
adequate or appropriate professional development, what does a working model look like?
Shaha, Glassett, and Copas seek to answer this question with a twenty seven state study
that aims to identify the “impact of teacher observations in alignment with professional
development on teacher efficacy,” which was quantified by analyzing two hundred ninety two
schools, operating in one hundred ten districts and more than two dozen states.
These three authors argue that the end goal of observation and evaluation within a school
setting is to provide the proper environment and interactions that “result in improved student
30
capabilities and achievements” (2015, p. 55). However, a significant amount of research has
shown that this relationship is strained and that schools chronically find the most appropriate
manner to “identify ways to continuously improve the impact teachers have on their students” (p.
55; Duke & Stiggins, 1990; U.S. Department of Education, 2013). The most common approach
to pursue the continued enhancement of instruction is professional development (Buczynski &
Hansen, 2010). A cannon of documentation and research bears light on the truth of the
effectiveness of professional development, however, which does not necessarily jive with the
expected findings. In spite of increased attention and focus on the role of professional
development in teacher improvement and support, “data substantiating improved impact of
teachers on students remains sparse” (p. 55; Avalos, 2011; Desimone, Porter, Garet, Yoon, &
Burman, 2002; Farnsworth, Shaha, Bahr, Lewis, & Benson, 2002; Garet, Porter, Desimone,
Birman, & Yoon, 2001; Lewis et al., 2003; Shaha, Lewis, O’Donnel, & Brown, 2004).
“Teacher observation is another technique widely used wherein school leaders, experts,
or designated coaches watch teachers and then provide feedback or guidance aimed at improving
impact for classrooms and student learning” (Shaha et al., 2015, p. 56). Many researchers since
the turn of the new millennium have indicated, nevertheless, that traditional teacher evaluations
fail to appropriately inform teachers of areas for improvement, rarely occur often enough to be
beneficial, and have not proved consistently effective beyond simply being a necessary action to
fulfill bureaucratic responsibilities (Darling-Hammond, 1986; Hazi & Arredondo Rucinski,
2009; Jacob & Lefgren, 2008; Peterson, 2000; Stiggins & Bridgeford, 1985; Weisberg et al.,
2009). Consequently, Shaha, Glassett, and Copas all argue in their writing that supervisory
observation as a means of teacher improvement provides only minimal or mixed outcomes that
do not drive toward improved student achievement (p. 56). It is in the pursuit of identifying best
31
practices of evaluation and observation (followed by appropriate professional development) that
these authors have conducted their research.
One of the most central findings of Shaha, Glassett, and Copas’s research is that, when
teacher observation is combined with appropriately matched professional development, student
performance in class improves (2015, p. 58). Similar research-based conclusions point to a direct
and positive correlation between the amount of time a teacher is observed in the classroom by
their supervisor and greater gains in student accomplishments on standardized math and English
assessments (p. 58). The recommendations based on this research point to the important relation
between a principal’s observation of a teacher and the pertinent recommendation of professional
development opportunities, whether in an on-demand, online program or through a more
traditional approach. Consequently, when schools and districts and states “work to improve the
education in their schools and in those throughout America and the broader educational world,
the central focus of validation and verification efforts must include the evaluation of teachers for
the purpose of improving their impact on students” (p. 60). And if schools seek to improve
student academic outcomes, it is important that, “although observation-based teacher evaluations
might be criticized and disconnected from the needs of students” research indicates that “a
coordinated approach involving [professional development] recommendations and executions is
impactful for student advances” (p. 60).
A new trend in education research seeks to identify how this customary approach to
observation (conducted solely by the principal or supervisor) can be supported and supplemented
by peer observation of teaching (Eri, 2014). In order to contextualize peer observation of
teaching, Eri, in his 2014 publication Peer Observation of Teaching, provides a definition for the
practice as a “reciprocal process where a peer observes another’s teaching” and then “provides
32
constructive feedbacks that would enable teaching professional development through the mirror
of critical reflection by both the observer and the observed” (p. 625; Brookfield, 1995). Eri
identifies five primary benefits of peer observation of teaching which include: (1) enhancing the
quality and skills of the observed teacher, (2) participants gain confidence in their teaching
methods, (3) new ideas are acquired for more effective instruction, (4) teaching methods and
resources can be shared more easily, and (5) it creates a mutually shared assurance of “continued
commitment to teaching” (Eri, 2015, p. 625). The author further identifies the procedure that
peer observation of teaching should follow in order to be most effective. The process begins with
a pre-observation, where the observer and observee identify the milieu of the observation,
including time and location, as well as the particular classroom practices to be observed. After
this initial meeting comes the observation itself, which should be for a more than merely cursory
amount of time and include detailed note taking and critical observation. This is followed by the
post-observation meeting where feedback is provided and a conversation is held that explores the
nature of the observation and the thoughts of both participants. Subsequently, both individuals
engage in a personal process of critical reflection, a semi-metacognitive effort, to truly think
about the teaching and observation and discern the value that has been borne of it. The final stage
is to implement any suggestions for change and improvement (p. 626–629).
This background on peer observation of teaching leads to a more in-depth discussion on
the topic of providing appropriate feedback to practicing teachers, whether this be from
colleagues or supervisors. In the 2011 research study “Teacher Conferencing and Feedback:
Necessary but Missing,” published in the International Journal of Educational Leadership
Preparation, Anast-May, Penick, Schroyer, and Howell seek to address a common concern. The
authors note that because “there is an absence of systematic feedback for teachers to facilitate
33
[the] professional growth [of teachers and to] improve instruction,” evaluators “tend not to
provide detailed and concrete feedback after they have observed teachers” and that “without
objective feedback and regular reports on progress and performance, an individual is less likely
to achieve his or her professional goals” (p. 2).
On the other hand, “constructive and meaningful feedback is needed to promote
reflection and allow teachers to plan and achieve new goals, which will ultimately lead to an
increased sense of efficacy in their teaching” (p. 2). The researchers have found that a significant
portion of supervisory feedback tends to be shallow, unhelpful, and inaccurate, rarely verging on
helpful or encouraging (p. 2; Frase, 1992). At the end of the day, it is argued, “an evaluation has
no meaning if it is not interpreted, questioned, discussed, and reflected on, ultimately leading to
making different and more effective decisions” (p. 2; Feeney, 2007).
Turnbull, Haslam, Arcaira, Riley, Sinclair, and Coleman (2009) identified that one of the
chronic shortfalls in the evaluation of teachers is that there is a disconnect between the amount of
time a teacher is observed and the amount of feedback received. These researchers also found it
common that “principals provided no individual feedback, choosing instead to focus on group
feedback based on a checklist criteria” (Turnbull et al., 2009). Two other researchers have found
that this is not merely an issue affecting the most inexperienced teachers, but that meaningful
feedback for teachers outside their probationary periods was almost non-existent (Kelley &
Maslow, 2005). This knowledge is what inspired Anast-May, Penick, Schroyer, and Howell to
conduct research that sought some answers as to why teachers tended to be skeptical about
increased observation, then provide recommendations for professional practice.
Although their research was on a more limited scale (three elementary schools with a
combined total of two thousand two hundred twenty five students), the research team was able to
34
safely conclude that “the process of evaluation should involve conferencing and feedback that
will lead teachers to construct their own understandings and set professional goals that are
measured in terms of student learning” (p. 6; Wheatley, 2005). Similarly, the researchers support
the notion that data collection should play a role in the effective evaluation of teachers, stating
that “measurement [should] be used from a deeper place of understanding, the understanding that
the real capacity of an organization arises when colleagues willingly struggle together in
common work that they find meaningful” and, additionally, that measurement provides “the kind
and quality of feedback that supports and welcomes people to step forward with their desire to
contribute, to learn, and to achieve” (p. 6).
But the incorporation of data into teacher evaluation does not solely come from the
analysis of student test scores on standardized assessments. Student surveys are becoming more
prevalent in determining the effectiveness of particular teachers (Liu et al., 2014). Liu and her
colleagues seek to answer a poignant question through research: “Does adding teacher and
student survey measures to existing measures (such as supervisor ratings and student attendance
rates) increase the power of principal evaluation models to explain across-school variance in
value-added achievement gains?” (para. 1).
The findings are starkly evident and reassuring. Utilizing a two-step multivariate
regression analysis, it was determined that incorporating “teacher and student survey measures
on school conditions to the principal evaluation model can strengthen the relationship between
principals’ evaluation results and their schools’ average value-added achievement gains in math
and in a composite of math and reading” (p. i).
A significant number of states, spurred on by their legislatures and state departments of
education, have begun to seek out multi-variable measures for teacher effectiveness. Many
35
districts have shown a keen interest in incorporating teacher evaluations conducted by students
(Illinois State Board of Education, 2011; Mattson Almanzán, Sanders, & Kearney, 2011;
National Conference of State Legislatures, 2011; Wacyk, Reeves, McNeill, & Zimmer, 2011).
This drive to incorporate student surveys follows a recognition of the need to bring in
more than simply one evaluator’s observations as the basis for determining an effective teacher.
New models being developed rely on multiple performance measures such as growth in student
achievement, leadership competency assessments, and school climate surveys. This creates a
more complete picture of principal effectiveness (Clifford, Behrstock-Sherratt, & Fetters, 2012;
Illinois Principals Association & Association of School Administrators, 2012; Mattson
Almanzán, Sanders, & Kearney, 2011; The New Teacher Project, 2012; Ohio Department of
Education, 2011; Roeber, 2011; Wisconsin Educator Effectiveness Design Team, 2011).
In the research conducted by Anast-May, Penick, Schroyer, and Howell, new information
is contributed to the discussion that schools and districts are currently having about whether to
include student surveys about teachers in the supervisory evaluation of teacher effectiveness. The
answers to this contemporary question were discerned by studying one particular school district
within the American Midwest region in depth during the 2011–2012 school year, encompassing
more than thirty nine total schools (2015, p. 2). The research sought to determine if student
surveys of teachers correlated in any way to standardized test scores in English, math, and
science, utilizing the Measure of Academic Progress (developed and administered by the
Northwest Evaluation Association) (p. 2). The student survey used was the Tripod Student
Perception Survey, which was developed by Harvard researcher Ronald Ferguson. This survey
“consists of thirty-six items measuring students’ perceptions of their classroom instructional
environment in seven domains and six items measuring students’ perceptions of school safety
36
and climate” (p. 3). Through the administration of this survey, it was confirmed—as numerous
previous studies have also shown—that the quality of the classroom teacher significantly impacts
student achievement (p. 6; Aaronson, Barrow, & Sander, 2007; Gordon, Kane, & Staiger, 2006;
Kane & Cantrell, 2010; Kane, Rockoff, & Staiger, 2006; Rivkin, Hanushek, & Kain, 2005).
What Anast-May et al. have been able to show is that this survey accurately predicts value-added
measures of teacher and school academic success on standardized tests, a novel finding (p. 7).
One of the primary supporting findings is the importance of school-level leadership on
student outcomes (p. A1–A2). In supporting literature, especially in a meta-analysis of more than
twenty existing research papers, it was determined that the “average effects of instructional
leadership practices on student achievement and other outcomes (such as absenteeism and
engagement) was three to four times as large as the average effects of other leadership practices
that do not explicitly focus on curriculum and instruction” (Robinson, Lloyd, & Rowe, 2008). By
measuring student perceptions of school leadership (which the aforementioned Tripod Survey
includes) it is possible to draw a direct connection between the efficacies of a school’s principal
and the outcomes that students are expected to achieve.
One state that has begun the process of implementing a multi-domain teacher
effectiveness evaluation program that includes both value-added measures as well as traditional
observation coupled with student survey responses is Arizona (Lazarev, Newman, & Sharp,
2014). This program of teacher evaluation began as a pilot project in five varying school districts
(four public and a charter network) throughout the state (p. i). Arizona chose to change its
teacher evaluation models in the wake of changes to the Elementary and Secondary Education,
Act, when states were required to submit various plans to the United States Department of
Education including their plans on how teacher evaluations needed to change to support a more
37
rigorous focus on instructional outcomes (U.S. Department of Education, 2012). Arizona was
one of the “two-thirds of U.S. states [that] have made changes to their teacher evaluation policies
since 2009” (Jerald, 2012).
Significant bodies of evidence have been compiled which have “yielded empirical
evidence of correlations between various teacher effectiveness metrics, including scores from
several widely used classroom observation instruments, student surveys, and estimates of
teachers’ value-added contributions to student test achievement” (Lazarev et al., 2014, p. 1; Kane
& Staiger, 2012). States, however, are often left with little existing research about how best to
integrate these varying evaluation tools in a meaningful way and to analyze and utilize collected
data (Rothstein & Mathis, 2013). It is to provide this empirical background for implementation
that Lazarev, Newman, and Sharp began to explore how this multi-domain evaluation system
was being implemented in the Grand Canyon State (p. 1).
The State of Arizona’s three domains of teacher evaluation include: (1) teacher
observation, (2) student academic progress, and (3) surveys (p. 2). Arizona chose to base their
state’s observation instrument on the Danielson Framework (Danielson Group, 2011). This study
chose to focus its effort on determining the relationship between the observation of teachers by
principals and the other two data collection instruments. And for good reason—one of the most
startling findings of the report is that the collected “results suggest that the observation
instrument was not used in a manner that effectively differentiated among levels of teacher
practice” (Lazarev et al., 2014, p. 5). Consequently, “there is evidence that observations by
school principals tend to produce inflated scores—in particular, many teachers with students
[who demonstrate] low academic progress receive high observation scores” (p. 5). Despite the
tendency of principals to over-praise a teacher’s effectiveness in the classroom, the researchers
38
were able to draw lines of positive correlation between the scores that a principal provided
utilizing the Danielson Framework and student academic achievement measures and surveys.
This means that the Arizona teaching framework (a replica of that produced by Charlotte
Danielson), consisting of twenty-two sub-domains for evaluation, can prove to be a statistically
validated measurement tool and, “if the observation items measure a single underlying aspect of
teaching effectiveness, a single aggregated observation score obtained by summing or averaging
item scores would be a valid measure of teaching” (p. 9).
Of the four domains of the Danielson Framework (planning and preparation, the
classroom environment, professional responsibilities, and instruction), the two areas that were
most highly associated with increased student achievement were the two that must be assessed
outside the classroom: planning and preparation and professional responsibilities (p. 17). Lazarev
and his colleague-researchers, then, cautioned any districts seeking to implement a multi-domain
evaluation tool to make sure that principals are well trained and know how to accurately evaluate
a teacher utilizing the most empirically-sound observation models (U.S. Government
Accountability Office, 2013). But the opportunity exists for strong adoption in other states.
The literature and history of teacher evaluation points to a continual endeavor to find the
most appropriate manner by which to evaluate an educator’s craft and effectiveness in the
classroom. It is a task that must be done, regardless of its challenges, because a failure to
appropriately evaluate a teacher is a failure to control and monitor and cultivate the best learning
environment possible for students in educational settings. As the MET Project (Measures of
Effective Teaching), funded by the Bill & Melinda Gates Foundation, succinctly points out:
“effective teaching is teaching that leads to student success” (MET Project, n.d.). In order to do
this most effectively, today’s educational researchers have begun to place a heavy emphasis on
39
finding reliable and valid measures of teacher effectiveness that strongly correlate to student
academic success; after all, “teaching evaluations are only valuable if they can be counted on to
accurately reflect what a teacher is doing in his or her classroom that is helping students
succeed” (MET Project, n.d.). Because teaching is recognized as a multi-faceted discipline, it
becomes evident that no single measure of success can accurately portray how effective a teacher
is performing in the classroom; for example, teaching takes significant amounts of planning and
preparation, and while circumstantial evidence might be present when a principal observes a
classroom, it can only be truly measured by other means.
The Implementation of Evaluation
It was the fourth century B.C. Greek philosopher Aristotle who, as quoted by Diogenes
Laertius in Lives of the Philosophers, stated that “the roots of education are bitter, but the fruit is
sweet” (Laertius, 1925). While the history of evaluation illustrates the manifold transformations
that supervisory observation has undergone, and the struggles between competing ideas and
ambitions, one end goal has remained: that a teacher evaluation must, in its final calculations,
lead to the most efficacious of teachers providing a powerful and transformative and
academically rich educational environment for students. But the path toward this final realization
is often fraught with the question of practical implementation, this point being illustrated with
the case of a disappeared Los Angeles teacher in 2010.
Rigoberto Ruela’s body was eventually found in the wooded slopes of Big Tujunga
Canyon (Zavis & Barboza, 2010). Twenty-six miles from the school he loved so dearly, where
he had taught fifth grade for fourteen years, the Angeles National Forest’s overgrown redwoods
could only hide Ruela’s body from the search-and-rescue team for so long before emancipating
40
its martyred educator. But this was not a hiking accident: the Los Angeles County Coroner
quickly ruled that the cause of death was suicide (Strauss, 2010).
The search-and-rescue team was only dispatched when Mr. Ruela failed to show up for
work at Miramonte Elementary School, a public school located in south Los Angeles, tucked
inside a community riddled with gang-violence and aching poverty (Zavis & Barboza, 2010). His
coworkers knew how dedicated Ruela was; the school’s administration confirmed that it was a
rare occasion that he did not show up to school—his record of attendance was nearly perfect over
his more than decade tenure—and never unannounced (Lovett, 2010).
His colleagues, however, could tell that he was slipping into a depression in the weeks
before his absence and death. In spite of the time that Mr. Ruela put into tutoring students after
school or communicating with parents (or even the complimentary review that he received from
his supervisor during the previous 2009/2010 school year), his family blames one particular
event for pushing him into a tailspin that ended with a base-jump from a bridge: a poor teacher
review and ranking, as calculated in a value-added manner by the Los Angeles Times and
published publically online, which rated him as a teacher who was “less effective than average”
(Romero, 2010; Lovett, 2010). Rigoberto’s brother Alejandro, in an interview with a Los
Angeles public radio station, announced that he does “blame the [Los Angeles] Times [for his
brother’s death] for the fact that they did publicize [his scores and ranking]… When you have the
L.A. Times labeling him that way that really hits you hard. He took his job seriously” (Romero,
2010).
This story’s beginning lies, however, back in the year 2010 when the Los Angeles Times,
the major metropolitan daily newspaper in Southern California, posed a bold question: “Who’s
teaching L.A.’s kids?” (Felch, Song, & Smith, 2010). Working with the Los Angeles Unified
41
School District, the newspaper evaluated the previous seven years of teachers’ impacts on
student test scores. (Abramson, 2010). These scores, designed to measure the effectives of
teacher instruction by a value-added measure, were supposed to be a non-biased, mathematical
means by which to judge the worth of a classroom instructor (Hancock, 2011).
Despite warnings and concern from researchers and leaders in education that the findings
provided by the Los Angeles Times were unreliable, the newspaper decided that it would publish
the data, and this was hailed as a monumental step by Arne Duncan, the then-U.S. Secretary of
Education, who encouraged other newspapers to follow the Times’s example (Anderson 2011;
Felch & Song, 2010). “What’s there to hide?” Duncan is quoted as saying the day after the Times
first published the teacher evaluation; “in education, we’ve been scared to talk about success”
(Freedberg, 2011). This message from the administration of the Barack Obama mirrored the
federal government’s push for localized value-added measurement of teacher effectiveness, as
spurred on by “Race to the Top.” This federal program sought to financially encourage states to
make it a legal requirement that teacher evaluations include, as their basis, quantifiable data on
how well teachers were improving student scores on standardized tests. It is a seemingly logical
evaluation, its developers argued, as any district should be willing to make a student’s growth in
knowledge a central component of determining a teacher’s value to a particular district and the
students he or she teaches (Strauss, 2012). The Association of Supervision and Curriculum
Development (a not-for-profit organization of principals, superintendents, teachers, and others in
the field of education) defines value-added assessment simply as measures that capture “how
much students learn during the school year, thereby putting teachers on a more level playing
field as they aim for tenure or additional pay” (David, 2010).
42
As the debate in Los Angeles heated up in the aftermath of Mr. Ruela’s death, the head of
the union of Los Angeles’s public school teachers said that these types of “value-added
assessments are a flawed system.” The Los Angeles Times subsequently released a statement of
its own, declaring both its sympathy for the death of the teacher, but also that
The Times published the database, which is based on seven years of state test
scores in the L.A.U.S.D. schools, because it bears directly on the performance of
public employees who provide an important service, and in the belief that parents
and the public have a right to judge the data for themselves (Lovett, 2010).
These two camps, on opposing sides of the debate, illuminate a larger trend in the evaluation of
teachers and schools, and the debate about which particular factors best fit into the calculus of
determining how effective a particular educator is in the classroom.
43
CHAPTER 3: METHODOLOGY
This school-community action project seeks to indigenize the Danielson Framework for
application at Little Wound School. This will be accomplished by surveying students, teachers,
and administrators from Little Wound School to identify internal beliefs about which
components of the Danielson Framework (adopted by South Dakota as the South Dakota
Framework for Teaching) are the most closely associated with what makes for a most effective
teacher. The collected information will be used to devise a uniform evaluation model for use
when principals and supervisors observe classroom teachers.
Subjects
In order to evaluate the school’s perceptions of the importance of the various components
that comprise the Danielson Framework for Teaching, it is important to identify the population
of individuals who were surveyed for their opinions. Three primary groups were identified to
take the survey: (1) students over eighteen years old, (2) teachers, and (3) administrators.
Because only students who are over eighteen years old will be surveyed, these students
will be in the twelfth grade. Only students at this age juncture will be surveyed to eliminate the
participation of vulnerable populations in the study, which includes youth below eighteen years
of age. While this limits the amount of student input, the presence of any of this population’s
input will still provide an opportunity for analysis. This group includes both male students (8)
and female students (19). All students surveyed were Native American (based on the designation
on the survey) and attended school between February 22 and February 26, 2016.
All teachers within the school between kindergarten and twelfth grade were provided a
survey; interventionist teachers were also included, as were teachers of specials in the elementary
school. (These two latter groups were included because they are subject to the same model of
44
evaluation by their direct supervisors.) The gender breakdown of teacher research participants
were male (19) and female (34). A majority of the teachers surveyed were Native American
(71%; 63 out of 88). The average tenure of teaching experience for a teacher-respondent was
10.8 years (median = 6), with some teachers having no prior teaching experience (zero years of
experience), while the most experienced teacher has taught a total of 34 years. Teachers were
surveyed the same week as the students.
Administrators were also surveyed. This group was the smallest of those surveyed, with a
total of eight individuals. This group included three principals (elementary, middle, and high
school), the superintendent, and other high level individuals. Of this group, seven are female and
one is male; all identify as Native American.
The following table, Table 1, identifies the rates of response from broken-out groups who
were surveyed for this research. (Information for the section of the table labeled “sample pool
size” is current as of the Friday of the week the survey was taken; all teachers and administrators
within the total pool of available survey-takers were provided with a survey.)
TABLE 1.
Rate of survey responses.
Sample pool
size:
Returned
surveys:
Percentage
completed:
Students
Over eighteen years of age 28 27 96%
Total: 28 27 96%
Teachers
Elementary (K–5) 20 18 90%
Middle school (6–8) 11 11 100%
High school (9–12) 24 24 100%
Total: 55 53 96.4%
Administrators
Total: 8 8 100%
45
In order to ensure privacy, no identifying information was collected from survey
respondents. Because of the sample size, it is impossible to track back one survey to the student
who took that particular survey. The privacy of teachers has also been considered, and no
identifying information appears within this report or on the surveys distributed to teachers.
Similar precautions apply to the administrators.
In order to confirm that the privacy and additional rights of all subjects were protected,
the research proposal for this report was submitted and approved by relevant area actors,
including the Oglala Sioux Tribe’s Research and Review Board, the Oglala Lakota College
Institutional Review Board, and the Little Wound School Board and superintendent.
Procedures and Data Collection
The Little Wound School Board approved this research project to be conducted through
the school itself—the research collected and the development of a model teacher evaluation in
compliance with state and BIE requirements will directly benefit the school and its teacher
observation and evaluation needs.
Consequently, all teachers in the school will be provided with a survey asking them to
identify, within each of the four domains of the Danielson Framework, the one component that
makes for a most effective teacher. Teachers will also be asked for their gender, their total years
of teaching experience, whether they are an enrolled member of a federally recognized tribe, and
if they have previously had any formal training on the Danielson Framework. In total, there will
be twenty surveys distributed to elementary school teachers—this includes all classroom
instructors, two Lakota language instructors, one physical education teacher, and two
intervention teachers: one English, one math. Within the elementary school, only these teachers
46
will be surveyed because it is only these staff members who are formally evaluated by their
direct administrators in compliance with state and Bureau expectations.
In the middle school, eleven teaching staff will be surveyed; this includes eight core-
subject teachers, two Lakota language instructors, and the instructor of the middle school’s
alternative education program. In the high school, the survey will be distributed to a total of
twenty classroom teachers. Like the elementary school, these staff members are formally
evaluated by their principal supervisors, and thus will be surveyed for their input about the most
central domain components to be utilized in a model teacher evaluation form.
A total of eight school administrators will also be provided with a survey. These
administrators will be asked to identify their number of years of administrative experience, their
total years of teaching experience, if any, and whether the survey-taker has had any training
provided by the originators of the Charlotte Danielson Framework for Teaching.
For all teachers and administrators, the survey—with an accompanying letter explaining
the purpose of the survey, the fact that it is voluntary, and that no identifying information will be
collected—will be placed in their individual mailboxes on a Monday morning before school
starts. (All teachers and administrators will be informed the week before by email that the survey
will be handed out the following week.) Teachers and administrators will fill out the survey
independently and submit the document to a sealed box located near all participants’ respective
mailbox clusters (one in the elementary school, one in the middle school, one in the high school,
one in the administrative offices). Staff members will have until the Friday of that week to return
their completed surveys for inclusion in the research project. In this manner, all returned surveys
will remain entirely anonymous and the non-identifying information will be held confidential.
47
Because of the inclusion of only students above eighteen years of age, students will be
selected in a non-randomized manner. These students will be provided the same survey as staff
and administrators with only some minor changes for the information collected, and will include
a letter of informed consent—like all prospective participants. These adult-aged students will be
instructed to return their completed surveys to the same locked box for high school staff
members. (These students will all be in the high school.)
Because the survey will be conducted at a school operated under the auspices of the
Bureau of Indian Education on a Native American reservation, it is an assumption of the
researcher that most respondents will identify as enrolled members of a federally recognized
Indian tribe. In order to confirm that this is, in fact, accurate, survey takers will be asked to
identify if they are enrolled. While this question could have been rephrased to ask if a participant
identifies as Native American, asking for confirmation of tribal enrollment is a method by which
to gather this information in the same manner that the U.S. Census Bureau conducts its
demographic research (Norris, Vines, & Hoeffel, 2012). Information collected is not limited to
enrolled tribal members, and the responses from individuals who are not enrolled members will
prove similarly enlightening when the data collected is analyzed.
The participants are all informed that the survey is entirely voluntary in nature and the
completion of the survey is not required. Similarly, in line with making sure that all the
participants taking the survey are engaging in the activity with informed consent, the letter
provided also points survey-takers to the lead researcher and research committee chairperson in
case of discontent, and also includes information on contacting the Oglala Lakota College and
tribal institutional review boards in case of concern. These letters of consent were attached to the
front of all surveys distributed, and were returned with completed surveys. In a similar manner to
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation
The Indigenization of Evaluation

More Related Content

Similar to The Indigenization of Evaluation

Woods_Internship+Log+2011-2013
Woods_Internship+Log+2011-2013Woods_Internship+Log+2011-2013
Woods_Internship+Log+2011-2013
Matthew Woods
 
Insidethe blackbox
Insidethe blackboxInsidethe blackbox
Insidethe blackbox
Emma Grice
 
Retaining Teacher Talent: The View from Generation Y (Report)
Retaining Teacher Talent: The View from Generation Y (Report)Retaining Teacher Talent: The View from Generation Y (Report)
Retaining Teacher Talent: The View from Generation Y (Report)
Amber Ott
 
Action Plan TemplateUse this template to assist you with develop.docx
Action Plan TemplateUse this template to assist you with develop.docxAction Plan TemplateUse this template to assist you with develop.docx
Action Plan TemplateUse this template to assist you with develop.docx
AMMY30
 
Harvard-IOP-Education-Policy-Fall-2015
Harvard-IOP-Education-Policy-Fall-2015Harvard-IOP-Education-Policy-Fall-2015
Harvard-IOP-Education-Policy-Fall-2015
Catherine Zhang
 
Pass institute checklist project
Pass institute checklist projectPass institute checklist project
Pass institute checklist project
reneeot
 

Similar to The Indigenization of Evaluation (20)

Minutes of the Tenth AMCOA Meeting, May 1st, 2012 -- Framingham State University
Minutes of the Tenth AMCOA Meeting, May 1st, 2012 -- Framingham State UniversityMinutes of the Tenth AMCOA Meeting, May 1st, 2012 -- Framingham State University
Minutes of the Tenth AMCOA Meeting, May 1st, 2012 -- Framingham State University
 
Counseling chomprehensive (REFRENSI)
Counseling chomprehensive (REFRENSI)Counseling chomprehensive (REFRENSI)
Counseling chomprehensive (REFRENSI)
 
Campus compact march 2016 conference (1)
Campus compact   march 2016 conference  (1)Campus compact   march 2016 conference  (1)
Campus compact march 2016 conference (1)
 
Woods_Internship+Log+2011-2013
Woods_Internship+Log+2011-2013Woods_Internship+Log+2011-2013
Woods_Internship+Log+2011-2013
 
Characteristic of high school performance
Characteristic of high school performanceCharacteristic of high school performance
Characteristic of high school performance
 
Tame edited research . education and planning
Tame edited research . education and planningTame edited research . education and planning
Tame edited research . education and planning
 
Insidethe blackbox
Insidethe blackboxInsidethe blackbox
Insidethe blackbox
 
Hawaii Jan 2018 - international education conference
Hawaii Jan 2018 - international education conferenceHawaii Jan 2018 - international education conference
Hawaii Jan 2018 - international education conference
 
Essay Task
Essay TaskEssay Task
Essay Task
 
Assessment for aais
Assessment for aaisAssessment for aais
Assessment for aais
 
Highlights From Future of Education - mSchool + DreamBox Learning
Highlights From Future of Education - mSchool + DreamBox LearningHighlights From Future of Education - mSchool + DreamBox Learning
Highlights From Future of Education - mSchool + DreamBox Learning
 
Retaining Teacher Talent: The View from Generation Y (Report)
Retaining Teacher Talent: The View from Generation Y (Report)Retaining Teacher Talent: The View from Generation Y (Report)
Retaining Teacher Talent: The View from Generation Y (Report)
 
Action Plan TemplateUse this template to assist you with develop.docx
Action Plan TemplateUse this template to assist you with develop.docxAction Plan TemplateUse this template to assist you with develop.docx
Action Plan TemplateUse this template to assist you with develop.docx
 
Secondary Analysis Of Qualitative Data
Secondary Analysis Of Qualitative DataSecondary Analysis Of Qualitative Data
Secondary Analysis Of Qualitative Data
 
Tesol08 Learning Outcomes
Tesol08 Learning OutcomesTesol08 Learning Outcomes
Tesol08 Learning Outcomes
 
Signature Assignment CUR 528: Planning a Needs Assessment
Signature Assignment CUR 528: Planning a Needs Assessment Signature Assignment CUR 528: Planning a Needs Assessment
Signature Assignment CUR 528: Planning a Needs Assessment
 
Harvard-IOP-Education-Policy-Fall-2015
Harvard-IOP-Education-Policy-Fall-2015Harvard-IOP-Education-Policy-Fall-2015
Harvard-IOP-Education-Policy-Fall-2015
 
Missing Piece Program Methodology
Missing Piece Program MethodologyMissing Piece Program Methodology
Missing Piece Program Methodology
 
Pass institute checklist project
Pass institute checklist projectPass institute checklist project
Pass institute checklist project
 
An Evaluation Of One District U27S Think Through Math Program And Curriculum ...
An Evaluation Of One District U27S Think Through Math Program And Curriculum ...An Evaluation Of One District U27S Think Through Math Program And Curriculum ...
An Evaluation Of One District U27S Think Through Math Program And Curriculum ...
 

The Indigenization of Evaluation

  • 1. THE INDIGENIZATION OF EVALUATION: DEVELOPING A TEACHER EVALUATION TOOL FOR LITTLE WOUND SCHOOL by Alexander John Mackey A School / Community Action Project submitted in partial fulfillment of the requirements for the degree of Master of Arts (Educational Administration) in the Graduate Studies Department Oglala Lakota College 2016 Committee Members Lisa White Bull, committee chair Dr. Geraldine Gutwein Phinette Little Whiteman George Apple
  • 2. ii Master’s Committee The members of the committee appointed to examine the school / community action project and the accompanying master’s thesis of Alexander John Mackey find it satisfactory and recommend it to be accepted. ______________________________________________ Lisa White Bull, committee chair ______________________________________________ Dr. Geraldine Gutwein ______________________________________________ Phinette Little Whiteman ______________________________________________ George Apple
  • 3. iii
  • 4. iv Copyright 2016 by Alexander John Mackey. All Rights Reserved.
  • 5. v Yupćećela Oyake Wauŋspewića-kiya pi kta ća uŋ Charlotte Danielson ća egle pi wowapi ća 1996 hehan oṫokaheya wowapi ki le kaġa pi. Woiyuŋge wowapi waŋ etaŋ owayawa el oiṫaŋcaŋ yaŋkapi na wauŋspewica-kiye na wayawa, iya waŋiyetu ake-śagloġaŋ sam iya pi ća hena tukte woecuŋ iyotaŋhaŋḣći wauŋspewica-kiye wopika ća yua’taŋiŋ kta ća kaḣniga pi. Le wowapi suta ki woiyuŋspewica-kiye wowaśi ecuŋpi ki iwaŋwicayaŋka pi kte ki uŋ ća le kaġa pi. Ṫaopi cik’ala owayawa ki wauŋspewića-kiye taŋyaŋ ecuŋpi iwaŋwićayaŋka pi wowapi waŋ kaḣniga pi.
  • 6. vi Abstract The Charlotte Danielson Framework for Teaching, first published in 1996, sought to create a uniform understanding of the characteristics that make an effective teacher by categorizing teacher behaviors into four domains of practice. This effort was in response to a fractured history of teacher evaluation systems in the United States. In 2015, the State of South Dakota published its requirements for instructional supervisors’ evaluations, based on the Framework for Teaching. Schools, in beginning the implementation process, are required to choose one component from within the Framework’s four domains. This research project was designed to determine which components should be utilized at Little Wound School, located on the Pine Ridge Indian Reservation. A survey was distributed to administrators, teachers, and students over eighteen years old to determine which components are the most reflective of an effective teacher. The data collected was analyzed using statistical tests to determine the existence of discernable predilections among respondents. This was confirmed. Across domains one, two, three, and four, components B, A, C, and F, respectively, proved to be preferred choices. Taking this information into account, a model teacher evaluation tool is proposed for Little Wound School.
  • 7. vii Acknowledgements This thesis would not be completed without the encouragement and support of those around me, especially Mr. Russell Childree and Mr. Jesus Fuentes. Similarly, I thank my advisory committee. Ms. Lisa White Bull’s support as the chair was a welcome foundation to my work, and her kind words and guiding suggestions have shaped this paper greatly. Similarly, I am thankful for Dr. Geraldine Gutwein and her discerning eyes, which have crafted this paper into a readable and presentable work while Ms. Phinette Little Whiteman and Mr. George Apple have influenced the orientation toward the Lakota Way. I thank my family for their work in preparing me as a writer. And I thank Ms. Taylor Christensen for spotting an errant “dana” that was meant to be “data.” This paper would not be the same without Mr. Jon Wenger—whose mathematical proclivities were central to the analysis of collected data—and the time that he spent teaching an English teacher to use SyStat and make sense of its output; this will be eternally appreciated. Finally, I thank all the students in my classes throughout this past academic year who have heard me talk about my time at Oglala Lakota College and my effort in writing this thesis. Regardless of if they wanted to hear about this school community action project, they did, and the confessional of my classroom was gallantly staffed by them.
  • 8. viii TABLE OF CONTENTS ABSTRACT (Lakota) ………………………………………………………… v ABSTRACT (English) ………………………………………………………… vi ACKNOWLEDGEMENTS ………………………………………………………… vii LIST OF TABLES ………………………………………………………… x LIST OF FIGURES ………………………………………………………… xi LIST OF APPENDICES ………………………………………………………… xii CHAPTER 1. INTRODUCTION ………………………………………………… 1 Statement of Problem Importance of Study Definition of Terms Limitations Delimitations Assumptions 2. HISTORICAL CONTEXTUALIZATION OF EVALUATION ……… 16 A History of Teacher Evaluation in the U.S. Contemporary Research and Findings The Implementation of Evaluation 3. METHODOLOGY …………………………………………………… 43 Subjects Procedures and Data Collection Data Analysis Summary 4. FINDINGS ………………………………………………………… 61 Response Rate Demographic Data Findings Statistical Symbols 5. DISCUSSION ………………………………………………………… 72 Summary Discussion Conclusions Recommendations
  • 9. ix 6. IMPLEMENTATION ………………………………………………… 83 BIBLIOGRAPHY ………………………………………………………… 89 APPENDICES ………………………………………………………… 107
  • 10. x LIST OF TABLES Table 1. Rate of survey responses ………………………… 44 Table 2. Responses for Domain 1 across groups (n = 88) … 50 Table 3. Responses for Domain 2 across groups (n = 88) … 53 Table 4. Responses for Domain 3 across groups (n = 88) … 55 Table 5. Responses for Domain 4 across groups (n = 88) … 58 Table 6. Rate of survey responses ………………………… 61 Table 7. Demographics of survey respondents ………… 62 Table 8. Statistical symbols ………………………………… 71 Table 9. Variance among student, teacher, and administrator responses ………………………… 76 Table 10. Percentage of responses for top component preferences ………………………… 79 Table 11. Recommended Danielson Framework components for effective teacher evaluation ………………… 81
  • 11. xi LIST OF FIGURES Figure 1. Danielson Framework for Teaching in comparison to the NBPTS’s Core Propositions ………………………………………… 26 Figure 2. Distribution of Domain 1 component preferences in relation to years of teaching experience … 52 Figure 3. Distribution of Domain 2 component preferences in relation to years of teaching experience … 54 Figure 4. Distribution of Domain 3 component preferences in relation to years of teaching experience … 57 Figure 5. Distribution of Domain 4 component preferences in relation to years of teaching experience … 59 Figure 6. Evaluation tool for Little Wound School ………… 83
  • 12. xii LIST OF APPENDICES Appendix A. Charlotte Danielson Framework for Teaching … 107 Appendix B. South Dakota S. B. No. 24: An Act ………………… 110 Appendix C. Determining Teacher Effectiveness in South Dakota ………………………………………… 111 Appendix D. Calculating Teacher Effectiveness ………………… 112 Appendix E. Key Excerpts from the South Dakota Teacher Effectiveness Handbook ………………………… 114 Appendix F. Teacher Effectiveness: State Requirements Checklist 115 Appendix G. Tripod Survey ………………………………… 117 Appendix H. Survey Provided to Little Wound School Administrators, Teachers, and Students ………… 119 Appendix I. Statistical Symbols ………………………………… 123
  • 13. CHAPTER 1: INTRODUCTION Melody Schopp, Secretary of the South Dakota Department of Education, expressed her frustration with the patchwork of teacher evaluation models profligate across the state to the legislature. During a public hearing in Pierre, the state’s capital, on the department’s plan to devise a new framework for evaluation, Schopp proclaimed that “we just never had clear expectations for what’s expected of teachers” (Gahagan, 2010). But it was not enough to merely develop a framework for the sake of having one. Schopp continued that “what I don’t want to have happen is, ‘We’ve adopted it, and it’s done.’ We really want to move into an era of good collaborative feedback and ‘How can I grow as a professional?’” (Gahagan, 2010). The reasons behind her support for developing a new teacher evaluation model became more evident as the hearing continued when she mentioned that more than ninety-five per cent of collected evaluations ranked the observed teacher as satisfactory or above (Guilfoyle, 2013). The panel concluded that any collected information from existing evaluations was essentially meaningless. As a result, South Dakota realized the need to devise a new model for statewide observation and evaluation; the state would also need to “take away the mystique of evaluations” and provide teachers with a clearer understanding of how their own professional responsibilities can be improved with the aid of their principals along with a more comprehensive system of evaluation and feedback from trusted and trained supervisors (Gahagan, 2010). Any developed framework needed to meet this primary goal: to provide a clear view of how to become a more efficacious teacher in the classroom, therefore improving classroom instruction itself. After the public hearing, Schopp said “you have to start somewhere” (Gahagan, 2010). During the eighty-fifth session of the South Dakota legislature both the House of Representatives and the Senate took up this charge and passed what came to be known as S. B.
  • 14. 2 24 (2010), “An act to establish standards for teaching, to require teacher evaluations, and to provide for the development of a model evaluation instrument.” The bill did not outline what the adopted framework for teaching should include, but instead established the mandate that the state board of education would need to “promulgate rules... to establish minimum professional performance standards for certified teachers in South Dakota public schools, and to establish best practices for the evaluation of the performance of certified teachers that may be used by individual school districts” (S. B. No. 24, 2010). (See Appendix A for S. B. No. 24’s full text.) The state board’s working group, composed of administrators, teachers, and community members, created an evaluation framework that ranked teachers as “distinguished, proficient, basic, or unsatisfactory, using equal parts qualitative and quantitative measures” with the end goal being to “make teacher evaluation more meaningful, giving school leaders better information about which teachers deserve to be rewarded, dismissed or coached up” (Verges, 2012). In an update to the annual meeting of the Associated School Boards and School Administrators joint convention in Sioux Falls, held August 9, 2012, the dean of the education school at the University of South Dakota, a co-chair of the framework development working group, expressed the importance of feedback in the state’s new evaluation model, saying that teachers need to become active participants in their own evaluations. “Sitting down with a teacher and saying, ‘Tell me how you know your students are learning,’ I don’t think is a bad question to ask,” Melmer said. “We think it’s time for teachers to be more reflective” (Verges, 2012). What makes observation work is the conversation that naturally follows, the panel argued, not the act of observation. These ideas and others were published in April, 2015, as the South Dakota Teacher Effectiveness Handbook: Requirements, Support Systems, and State Model Recommendations,
  • 15. 3 the consequence of four years of internal development, community input, and work with local education agencies—including schools, districts, and leaders—across the state. Published by the South Dakota Commission on Teaching and Learning, this document outlined the state’s practical guidelines for implementing the group’s supervisory evaluation program. The state decided to base its observation model on an established guideline first organized and published in 1996 by Charlotte Danielson, who developed her framework for teaching while working for Educational Testing Service (ETS), the world’s largest maker and administrator of standardized assessments (Mercer, 2013; Singer, 2013). The Charlotte Danielson Framework for Teaching. As it has come to be known, the Danielson Framework (throughout this writing, the phrase “Danielson Framework” is used interchangeably with variations including “the Danielson Model,” the “Framework for Teaching,” or “Framework for Effective Teaching,” for instance) was chosen to serve as the state model for observation, evaluation, and feedback. Outlined in Enhancing Professional Practice: A Framework for Teaching, the book first published in 1996 by Charlotte Danielson, her framework is divided into four primary domains—these include planning and preparation, the classroom environment, instruction, and professional responsibilities—each comprised of between five and six components, which serve to add depth and provide a more concreate understanding of what the four domains include. The components are broken down into seventy- two elements that further elaborate on the components. For example, within the first domain, “planning and preparation,” the first component is 1a: “demonstrating knowledge of content and pedagogy,” which itself is sub-divided into three elements: “knowledge of content and the structure of the discipline,” “knowledge of prerequisite relationships,” and “knowledge of
  • 16. 4 content-related pedagogy” (Danielson, 2007). (For a listing of the Danielson Framework’s entirety, see Appendix A.) Danielson postulates that a “framework for professional practice” is important “because teaching is complex, [and therefore] it is helpful to have a road map through the territory, structured around a shared understanding of teaching” (p. 2). The Danielson Model is important, its developer argues, because “a framework for teaching offers educators a means of communicating about excellence.” As such, when “conversations are organized around a common framework, teachers are able to learn from one another” and the framework’s domains, components, and elements “are validated for any particular setting” (p. 6). South Dakota has adopted the Danielson Framework under its own moniker: the South Dakota Framework for Teaching, which is an exact reproduction of Danielson’s work (South Dakota Commission on Teaching and Learning, 2015, p. 10). Districts within the state are not required to utilize this particular measurement framework, but any district seeking an exemption must provide their own model that is directly aligned to the South Dakota Framework for Teaching. The State of South Dakota further notes that while the framework “includes twenty- two individual teaching components clustered into four domains,” district “effectiveness systems must include professional performance evaluations based on a minimum of four teaching components, including one from each domain” (p. 10). The Commission on Teaching and Learning, however, recommends that eight components be used as the basis of teacher observation and evaluation; it is the purview of the local education agency to determine which components are the most beneficial to the development of strong teachers (p. 16). Based on these domains and their components, and being coupled with student learning outcomes on state, standardized, and district assessments, the framework is designed to provide a
  • 17. 5 particular evaluation outcome: principals derive a Summative Teaching Effectiveness Rating that leads to a teacher being identified as below expectations, meeting expectations, or exceeding expectations. This summative teacher rating “includes an evaluation of student growth that serves as one significant factor and an evaluation of professional practices as the other significant factor” (p. 12). (See Appendix C for an outline for how effectiveness is measured; see Appendix D for a model provided by the state on how to calculate a teacher’s effectiveness score.) According to a timeline for implementation, the as-designed South Dakota Framework for Teaching will be fully implemented in public schools in the state for the 2015–2016 school year (South Dakota Commission on Teaching and Learning, 2015, Appendix B). Statement of Problem This research project seeks to identify what those elements of instruction are in a single school located on the Pine Ridge Indian Reservation, which is in southwestern South Dakota and home to the Oglala Sioux Tribe: Little Wound School. As the State of South Dakota and Bureau of Indian Education begin to require that schools adopt a teacher evaluation tool based on the Charlotte Danielson Framework for Teaching, it is necessary that local education agencies identify aspects of the framework most pertinent to teaching and learning in a classroom setting at each unique school site. Furthermore it is especially true for a school that serves primarily Native American students. Because schools have the leeway to decide how many and which components of the Danielson Framework will be evaluated (so long as it meets the minimum requirements of observation outlined by the South Dakota Department of Education) by supervisors, it is important that decisions made in this process be well informed and data-driven. In a search of literature, no published, peer-reviewed journal articles expressly sought to apply the Danielson
  • 18. 6 Framework to a tribally operated school, but existing research has shown time and again that students of diverse demographic backgrounds have inimitable educational needs (Koon, Petscher, & Foorman, 2014). This school-community action project sets out to determine which components of the Danielson Framework make the most effective teacher, as identified by students, teachers, and administrators at Little Wound School. With this information in hand, results can be reviewed, trends tracked, and proposals for practice proffered. Up to this point, data has not been collected from the Little Wound School District that answers this question. Through this collection process, answers to the following questions will be sought: 1. Will students, teachers, and administrators have different or similar views about the components of the Danielson Framework that make for a most effective teacher? 2. Can variations in the data collected from teachers and administrators be attributed to differences in the length of teaching experience, gender, position of employment, or other identifiable factors? 3. Will the results from students mirror or juxtapose those collected from teachers and administrative staff? 4. Are there particular components of the Danielson Framework that are unanimously believed by those individuals surveyed to be the most important factor in identifying a most effective teacher? It is the belief of the researcher that within each of the four domains of the Danielson Framework, one particular component will stand out as the most often cited factor in making for a most effective teacher. The information collected from this process will be used to guide the creation of a proposed evaluation tool for supervisors and principals to use as they observe
  • 19. 7 teachers and classrooms for instructional effectiveness. This proposed evaluation tool will reflect the beliefs and views of teachers throughout all grade levels of the school, principals and other administrators, as well as a sample of adult-aged students. Importance of Study This school-community action project seeks to indigenize the Danielson Framework for application at Little Wound School. By explicitly working to identify which components of the framework are viewed by students and staff as the most applicable in determining teacher effectiveness in this one particular setting, the Danielson Framework shifts from being an imposed method of evaluation to becoming a locally influenced tool for gauging teacher effectiveness. This process allows Little Wound School to utilize its own people’s knowledge in discerning this important measure. Schools that operate within the province of the Bureau of Indian Education (B.I.E.), of which Little Wound School is included, have for the past seven years been encouraged to utilize the Danielson Framework to evaluate teacher effectiveness. But the B.I.E.’s requirement to utilize the Danielson Model comes within a larger group of recommendations as part of the NativeStar regimen of “rapid improvement indicators” and “transformation implementation indicators” (Bureau of Indian Education, 2015). These different indicator groups (there are ninety-nine individual indicators for school improvement in the former group and twenty-four in the latter) only allude to the importance of utilizing the Danielson Framework as a foundation for classroom and instructional improvement. Each of these indicators are termed a “WiseWay.” For example, in WiseWay indicator forty one—“Indicator: The principal maintains a file of the agendas, work products, and minutes of all [instructional] teams”—schools are specifically directed to observe Danielson’s encouragement that “teacher evaluation should serve as an
  • 20. 8 opportunity for professional development,” (Academic Development Institute, 2013a). Likewise, in WiseWay indicator one-hundred-four—“Indicator: Yearly learning goals are set for the school by the Leadership Team utilizing student learning data”—Charlotte Danielson’s seminal book, Framework for Teaching, is referenced as being an important tool in being able to both understand and evaluate “multiple methods of assessment to engage learners in their own growth, to monitor learner progress, and to guide the educator’s and learner’s decision making.” (Academic Development Institute, 2013b). In totality, out of the collection of indicators, the works and ideas of Charlotte Danielson are directly referenced ten times. Consequently, in order for a Bureau of Indian Education grant school to fully adopt the NativeStar improvement indicators, it is central that schools adopt the Danielson Framework as a matrix by which to evaluate teachers and learning. All schools that operate within the Bureau of Indian Education are expected to adopt a NativeStar-based school improvement mindset. Once this is established, it is up to internal school policy-makers to determine how this is done—the Bureau does not provide additional direction. Schools are encouraged to look at their home state’s recommendations. For Little Wound School, this is South Dakota. More specifically, it is the state’s report, the South Dakota Teacher Effectiveness Handbook: Requirements, Support Systems, and State Model Recommendations. This monograph additionally requires schools utilize the Danielson Framework (or the “South Dakota Framework for Teaching,” whose content is the same) for teacher evaluation. Consequently, Little Wound School—on both the state and federal side—is being prodded to adopt this framework. Little Wound, in the first years of implementation, need only evaluate teachers on at least one component in each domain—four total. This school has yet to establish a standard to be used
  • 21. 9 across all schools (elementary, middle, and high) by all supervising administrators. While the school may wish to seek established recommendations about which components to choose, no existing research has been conducted that indicates which components are most efficacious to determine this information; similarly, there have been no internal research activities or surveys to determine what currently employed staff members and enrolled students believe to be the components that elucidate this information. The development of a teaching evaluation framework for Little Wound School is important because not only is it a requirement of the State of South Dakota and the Bureau of Indian Education, but little research has been conducted about what elements of teaching are the most important in determining effectiveness in a Native American school setting, especially when aligned to the Danielson Framework, which this research seeks to do. While some research indicates ways that Native American students generally learn best in a classroom setting (Tharp & Yamauchi, 1994; Morgan, 2009), no discernable research has been conducted about what Native American educators, students, and administrators believe are the traits that lead to effective teaching, especially when considered on the Pine Ridge Indian Reservation, the home of Little Wound School. This research project aims to unveil some of these individual perceptions and yield a teacher evaluation framework that is indigenous to Little Wound School and its students and staff. Definition of Terms In order that this thesis be understood in its entirety, it is important to define some key terms used throughout the paper. These definitions are curated from selected publications and reviews and articles that provide the most complete sense of the word’s true contextual meaning.
  • 22. 10 Achievement and Learning: Student achievement is “the status of subject-matter knowledge, understandings, and skills at one point in time” while student learning is “growth in subject-matter knowledge, understandings, and skill over time. In essence, a change in achievement constitutes learning. It is student learning—not student achievement—that is most relevant to defining and assessing accomplished teaching” (Linn, Bond, Carr, Darling- Hammond, Harris, Hess, & Shulman, n.d., p. 28). Clinical supervision: A process of supervision with three features: (1) autonomy: “the goal is for the teacher to become more self-directed and analytical,” (2) evidence: “the evidence for change in behavior arises from the observational data,” and (3) continuity: “the process unfolds over time” (Kilbourn, 1982). Domain: A particular “sphere of activity or knowledge” (“domain,” 2015). Within proper context, a domain is also one of four areas of supervision as identified in the Danielson Framework, which include: planning and preparation, classroom environment, instruction, and professional responsibilities (Danielson, 2007). Effective: This term is used to describe a teacher who has been “the most successful in helping [students] to learn” information and content (Walker, 2008, p. 63). Evaluation: A process to determine various aspects of teaching (a teacher’s effectiveness) that includes three essential elements: (1) “a coherent definition of the domain of teaching (the ‘what?’), including decisions concerning the standard for acceptable performance,” (2) analysis of “techniques and procedures for assessing all aspects of teaching (the ‘how?’),” and (3) the process is done by “trained evaluators who can make consistent judgments about performance, based on evidence of the teaching as manifested in the procedures” (Danielson & McGreal, 2000).
  • 23. 11 Improvement: A progressively unfolding process whereby achievement in a particular subject or area gets better in a measurable manner; it does not “have a fixed or predetermined end point, and that is sustained over extended periods of time” (Hidden Curriculum, 2014). Instruction: From the Association for Supervision and Curriculum Development, a definition of instruction is provided: “Instruction consists of any steps taken in planning and conducting programs of studies and lessons that suit them to the… students’ learning needs, learning readiness, and learner characteristics” (Heathers, 1977, p. 342) Model / framework: Used interchangeably, broadly, it is a “system of concepts, assumptions, expectations, beliefs, and theories that supports and informs your research” and practice (Miles & Huberman, 1994). Specifically, in the field of education, within a context of supervision and evaluation, it is a “blueprint for implementation” of a particular theory or practice to improve classroom instruction (California Department of Education, 2015). Observation: As provided by the South Dakota Commission on Teaching and Learning, there are two distinct types of observations: formal observations, which are “scheduled observation[s] of teaching practice conducted by an evaluator that is at least 15 minutes in length and includes structured conversations before and after the observation takes place” and informal observations, which is “an observation of teaching practice, which may or may not be announced, that is conducted by an evaluator, and is at least five minutes in length, and results in feedback to the teacher” (2015, p. 38). Professional development: This term means a “comprehensive, sustained, and intensive approach to improving teachers’ and principals’ effectiveness in raising student achievement” that “is aligned with rigorous state [and local] student academic achievement standards,” “is conducted among learning teams of educators, including teachers, paraprofessionals, and other
  • 24. 12 instructional staff at the school,” and finally “engages established learning teams of educators in a continuous cycle of improvement” (National Staff Development Council, n.d.). Value-added model: An assessment that analyzes “test data that can measure teaching and learning” which is based on “a review of students’ test score gains from previous grades” in order to determine whether students have “made the expected amount of progress, have made less progress than expected, or have been stretched beyond what they could reasonably be expected to achieve” (University of Pennsylvania, 2004). Limitations Limitations within this school-community action project are beyond the control of the researcher but may impact the study’s overall worth. Although this research was carefully planned, organized, analyzed, and presented, some of the limitations that affect this particular thesis include the following: First, not all teachers have had professional training on the Charlotte Danielson Framework and, consequently, do not have a full understanding of the material surveyed. In the spring of 2013, Little Wound School hosted a full-teaching staff professional development with a representative from the Danielson Group and all teachers were given a copy of Enhancing Professional Practice: A Framework for Teaching. No teachers have been formally trained on the framework in the three subsequent years and no further books have been given out. Teachers who became employed by Little Wound School in subsequent years may not have ever been exposed to the Danielson Group’s Framework for Teaching. Also, the population surveyed was narrow—staff, administration, and some students at Little Wound School during the 2015/2016 school year—and therefore cannot be immediately generalized beyond this one particular local education agency. Results or suggestions for
  • 25. 13 implementation may not be generalizable, even to schools that share similar demographics or environments. Not all students at the school were surveyed—only students over the age of eighteen shall be participants in this research. This eliminates the study’s work with a vulnerable population (below eighteen years of age) while simultaneously permitting the researcher to provide identically worded survey to all participants and, subsequently, removing concerns about the reading level of the material provided. This group will still provide valuable information. It must also be noted that the data collected is a snapshot of one particular point in time. Each individual who completed the survey will only take the survey once. Consequently, circumstances within the school environment may change and, as a result, impact people’s perceptions of supervisory observation and evaluation. For example, a change in school leadership or within the school board could see a reevaluation of teacher evaluation, new focuses in what to evaluate, and new metrics by which to gauge how effective individual teachers are. Similarly, as teachers come and go, overall staff perceptions may shift or change, endangering the long-term usability of this information. Finally, any identified correlation does not equal causation. Even if all the individuals surveyed provided the same responses, there could be no direct line drawn between what people think makes for a most effective teacher and what truly does make a teacher great. Delimitations Delimiting factors fall within the control of the researcher and could impact the quality of the work collected. In pursuit of transparency, some of the most important decisions that could produce altered results are included below. These decisions have been carefully weighed,
  • 26. 14 however, and were made with the intention of reducing errant outcomes and minimizing the opportunity that research collected cannot be used. First, only individuals employed by or attending Little Wound School at the time the survey was conducted were eligible to be included in the research. Second, on the survey, reference was made to the Danielson Framework for Teaching, which is often associated with evaluation and observation, but on the survey utilized, the question posed for the individual answering survey read: “What is the most important element in each box that makes for the most effective teacher?” The Danielson Framework is designed to measure excellence in teaching, which warrants the identification of the effectiveness of particular teachers. If teachers were asked which of the components of the Danielson Framework are most important in a principal’s observation of a classroom, the information might merely reflect what is observable, what is base, and (potentially) what is easiest. It should also be noted that a survey was the sole method of information collection for this research. There is no included ethnographic element, and the methodological evaluation is entirely quantitative. It should be noted, however, that because of the sample size, data collected is legitimate and can be utilized to create a localized teacher evaluation system. Finally, the content of the survey distributed utilizes the same wording and phrasing as developed by Charlotte Danielson. The vocabulary is of a high level and some participants might not be fully versed in the vocabulary. The decision has been made to maintain this wording because it provides the most accurate reflection of the school’s present understanding of the Framework as well as maintains Danielson’s original intent.
  • 27. 15 Assumptions The following assumptions are made regarding the research for this writing and for the continued use of any data collected or patterns discerned. A first major assumption of this thesis is that Little Wound School will, per the directive of the Bureau of Indian Education, continue to utilize the Danielson Framework as the primary tool of classroom observation and teacher evaluation. Should policy change at the national, state, or local level, and the Danielson Framework were to be replaced or modified, the survey and the results collected and analyzed herein would be moot. This survey also anticipates that answers collected are based on truthful and honest responses from the survey participants.
  • 28. 16 CHAPTER 2: HISTORICAL CONTEXTUALIZATION OF EVALUATION A History of Teacher Evaluation in the United States Evaluation of teachers in the United States can be tracked back to the days that schools were first being established in disparate communities in the 1700s. At this time, there was no unified professional discipline of education, and responsibility for both founding and monitoring schools and teachers fell on the town in which it was located and upon the parents whose children attended that particular school; the community often appointed an individual or established a supervisory committee that set the standards by which a teacher served the town (Burke & Krey, 2005). As a result of this lack of pedagogical consistency, the quality of teachers could vary greatly from place to place and school to school. This school-by-school variation began to coalesce into centralized administration in the 1800s with the “common school” movement, whose biggest proponent was Horace Mann, the first secretary of the State Board of Education in Massachusetts (Simpson, 2004). While these informal early school districts had a superintendent, the burden of administration typically fell upon a ‘principal teacher,’ who, in time, become simply the ‘principal,’ as the title is understood today (Marzano, Frontier, & Livingston, 2011). As schools become more professionally managed, a debate began to arise in the late 19th and early 20th century—education’s first crisis of purpose. One of these two competing views of education was spearheaded by John Dewey, who proposed that democracy was the true underpinning of societal progress and that the purpose of schools can only properly be measured by how well they encouraged civic action and participation in civil society (DeCesare, 2012). In Dewey’s view, “Progressive ideas such as a student-centered education, connecting the classroom to the real world, differentiation based on student learning needs, and integration of
  • 29. 17 content areas were espoused” by the thinker as “ways of bridging the gap between students’ passive role as learners and the active role they would need to play as citizens” (Marzano et al., 2011). The opposing camp was manifested by Frederick Taylor, who believed that any organization must find the most effective manner by which to accomplish their goals, then universalize and standardize that particular practice (Freedman, 1992). This latter philosopher’s ideas, in the early 20th century, came to embody how schools evaluated their success. Pushed forward by the writings of a Harvard professor, Edward Thorndike, and applied to education by Ellwood Cubberley, schools evaluated teachers in a same way that factories measured their own productivity. In Cubberley’s 1916 book Public School Administration, the author states that: Our schools are, in a sense, factories in which the raw products (children) are to be shaped and fashioned into products to meet the various demands of life. The specifications for manufacturing come from the demands of twentieth century civilization and is the business of the school to build its pupils according to the specifications laid down (p. 338). It was from this factory-model of education that Cubberley established his understanding for how school success should be measured, which emphasized “measurement and analysis of data to ensure that teachers and schools were productive” (Marzano et al., 2011). These two competing views stood in opposition to one another for decades to come, reinforced by various philosophers, academics, and school leaders. But as time progressed, individuals within the educational field began to see this as a false dichotomy; the views of Taylor and Cubberley provided a feedback loop that encouraged teachers to be more effective, leading students to a greater involvement in civil society, which was the idea that Dewey
  • 30. 18 espoused more than a century earlier (Raths & McAninch, 2005, p. 164). It is from here that a school’s dual purpose, which became more pronounced in the post-World War Two era, become evident: that schools must be data-driven institutions in pursuit of broad goals established by society and the community (Marzano et al., 2011). It is this two-pronged approach that gave rise to one of the most widely adopted methods of school and teacher evaluation: clinical supervision. Coming into the forefront of educational thought in the 1950s, clinical supervision is best known as culminating to and emanating from a 1969 book written by Robert Goldhammer, Clinical supervision: Special methods for the supervision of teachers; this book established the basic need of school leadership to maintain close communication with their teachers, observing their practice, and providing appropriate and immediate feedback for improved instruction (Goldhammer, 1969). By 1980, surveys found that more than ninety percent of school leaders used Goldhammer’s approach to school evaluation in their practice (Marzano et al., 2011). A 1973 work by Morris Cogan—Clinical Supervision—further elaborated on the intricacy of the practice, stating that: A cornerstone of the supervisor’s work with the teacher is the assumption that clinical supervision constitutes a continuation of the teacher’s professional education. This does not mean that the teacher is “in training,” as is sometimes said of preservice programs. It means that he is continuously engaged in improving his practice, as is required of all professionals. In this sense, the teacher involved in clinical supervision must be perceived as a practitioner fulfilling one of the first requirements of a professional—maintaining and developing his competence. He must not be treated as a person being rescued
  • 31. 19 from ineptitude, saved from incompetence, or supported in his stumblings. He must perceive himself to be engaged in the supervisory processes as a professional who continues his education and enlarges his competences (p. 21). The clinical supervisory model, however, had its downfalls and detractors: it was considered “didactic” and “formulaic,” and that “supervisory and evaluative approaches that were more developmental and reflective were sometimes viewed as not specific enough to enhance pedagogical development” (Wise, Darling-Hammond, McLaughlin, & Bernstein, 1984). It was believed among surveyed teachers that there were four primary issues with the clinical observation model of teacher evaluation: (1) principals did not have the “resolve and competence” to accurately evaluate teachers; (2) teachers, overtly or covertly, resented receiving critical feedback; (3) there was no uniform evaluation measures by which standards of instruction could be applied to all teachers evenly; and (4) was that little or no training was available for principal-evaluators, which led to inconsistency in implementation (Wise et al., 1984, p. 22). From these critiques rose a seminal work, written by Charlotte Danielson in 1984: Enhancing Professional Practice: A Framework for Teaching. Danielson’s background was working for Educational Testing Services (ETS), and in particular working on identifying measurable ways by which to effectively “measure the competence of teachers” (Marzano et al., 2011). Historical models of evaluation focused on singular aspects of the teacher’s actions, be it the steps of the teaching process or the supervisory process, for instance, but Danielson sought to create a comprehensive model for supervisory observation that captured the dynamic processes involved in classroom teaching.
  • 32. 20 The Danielson Framework, as it has come to be known, is composed of four domains: Planning and Preparation, the Classroom Environment, Instruction, and finally Professional Responsibilities; each of these domains is then broken down into subdomains, seventy six in all, which seek to work in tandem and provide a common language of evaluation for all stages in the teaching process, from planning to reporting of data (Danielson, 1984). The twenty-first century saw a modification of Danielson’s Framework in an effort to incorporate a more student-centered view. This new paradigm was championed by Tucker and Stronge and outlined in their 2005 book Linking Teacher Evaluation and Student Achievement, which advocated the “importance of student achievement as a criterion in the evaluation process” and “argued for evaluation systems that determine teacher effectiveness using evidence from student gains in learning as well as observations of classroom instruction” (Marzano et al., 2011). It is the work of these two individuals that is accredited with the popular rise of value- added (quantitatively-based) teacher evaluations, similar to the ones adopted and published by the Los Angeles Times in 2010 and 2011. The authors forcefully state in their recommendations that, “given the clear and undeniable link that exists between teacher effectiveness and student learning,” this should “support the use of student achievement information in teacher assessment. Student achievement can, and indeed should be, an important source of feedback on the effectiveness of schools, administrators, and teachers” (p. 102). In the last decade, many of the models of teacher evaluation have come under siege and are chided as “superficial, capricious, and often don’t even directly address the quality of instruction, much less measure students’ learning” (Toch & Rothman, 2008, p. 1). Toch and Rothman, in their 2008 study Rush to Judgement, found that, in spite of No Child Left Behind’s requirements for more rigorous forms of teacher evaluation, only fourteen states, at the time,
  • 33. 21 even required a single observation of a principal for a teacher; many of these evaluations only required supervisors to check boxes that teachers were “satisfactory” or “unsatisfactory” (Toch & Rothman, 2008). Additional contemporary reports found similarly troubling findings that demonstrated that a rigorous form of teacher observation and evaluation was still nothing more than an ethereal end-point. Research that took the authors to twelve school districts across four states—data was collected including on more than fifteen thousand teachers, one thousand three hundred administrators, and nearly one hundred district-level administrators—identified trends that were troubling portents illustrating deep flaws in teacher evaluations in major American school districts (Weisberg, Sexton, Mulhern, & Keeling, 2009). These researchers found that school districts tend to assume that classroom effectiveness is: the same from teacher to teacher. This decades-old fallacy fosters an environment in which teachers cease to be understood as individual professionals, but rather as interchangeable parts. In its denial of individual strengths and weaknesses, it is deeply disrespectful to teachers; in its indifference to instructional effectiveness, it gambles with the lives of students (p. 4). Weisberg, Sexton, Mulhern, and Keeling, the writers of the report (who were assisted by contributing authors including Schunck, Palcisco, and Morgan) propose that the history of teacher evaluations is fraught with failures of understanding; inconsistent application of evaluations by principals, schools, districts, and states; a false understanding of the differences in instructional styles of teachers; and too short and too infrequent to be beneficial. All of these results, in the end, create a system where “excellent teachers cannot be recognized or rewarded, chronically low-performing teachers languish, and the wide majority of teachers performing at
  • 34. 22 moderate levels do not get the differentiated support and development they need to improve as professionals” (p. 6). It is clear, the report concludes, that the current system of evaluation is not a practical or useful model and must be abandoned for something more useful and responsive. Contemporary Research and Findings In order to gain a more thorough understanding of contemporary forms of observation and evaluation, it is necessary to abandon a general historiographical approach in favor of a more sharpened approach—it is necessary to analyze recent research reports that look at what, in fact, these types of evaluative and supportive relationships between principal and teacher look like in practice and in the field of education itself. Consequently, in the pursuit of a modern view of educational observation, it is central that only research from the twenty-first century be incorporated into a review of literature on this particular topic. Before exploring how to measure an effective teacher, though, it is important to look at how an effective teacher is being defined. In a 2008 report by Robert Walker, entitled Twelve Characteristics of an Effective Teacher: A Longitudinal, Qualitative, Quasi-Research Study of In-Service and Pre-Service Teachers’ Opinions, the loaded term “effectiveness” in education is defined as a “particular teacher who had been the most successful in helping respondents to learn” (p. 63). While broad in scope, it permits Walker to discern twelve characteristics that are generally associated with the most effective classroom instructors: being prepared; maintaining a positive attitude; holding students to high expectations; encouraging creativity; treating students equitably and fairly; displaying a “personable, approachable touch with students”; cultivating a sense of belonging among students; dealing with students compassionately; the possession of a sense of humor; demonstrating respect; being forgiving; and the admission of mistakes (p. 64).
  • 35. 23 Beginning in the year 2002, a not-for-profit organization, Teach For America, which takes primarily recent college graduates and, with one month of training, places these individuals into low-income, high-need communities, began to study its internal data in the pursuit of what makes for phenomenal teachers, whose students often grow by more than one and a half years within the timeframe of a single academic year (Ripley, 2010). This research was led by a chief researcher within the organization, Stephen Farr, who published a book with many of his findings in 2010: Teaching as Leadership: The Highly Effective Teacher’s Guide to Closing the Achievement Gap. In the book, Farr boldly claims that his primary finding is that “strong teachers insist that effective teaching is neither mysterious nor magical. It is neither a function of dynamic personality nor dramatic performance” but instead can be clearly delineated in a style that can be applied to many teachers (Farr, 2010). (Farr is careful to qualify his findings in that this is not the only way to achieve effectiveness in classroom instruction.) In his book, Farr outlines concrete and actionable examples that were common traits in high-level classrooms. These findings include the presence of “big goals” that drive classroom teaching and learning, consistent and meaningful checks of understanding, building investment with students and their families, purposeful planning, and continually seeking personal and professional growth, among others (Farr, 2010, p. vii–ix). Many of these findings were independently confirmed by a Mathimatica Policy Research report, which not only found similar common traits among effective teachers, but also that students in classrooms with Teach For America trained and supported teachers, “significantly outperformed those [classrooms led by] their more experienced counterparts” (Decker, Mayer, & Glazerman, 2004; Ripley, 2010). One of the selling points of Teach For America for prospective school districts seeking to hire its participants, or “corps members,” is that the organization provides a significant amount
  • 36. 24 of classroom support (including observation, evaluation, suggestions, and professional development) for teachers that supplements what schools and districts seek to do with their principal’s supervisory responsibilities (Cody, 2012). Teach for America seeks to employ multiple measures of student and teacher growth to develop a holistic, but nonetheless data- backed, system of measurement to determine effectiveness (Sawchuk, 2009). This multifaceted approach to teacher observation and evaluation has been practiced in Pittsburgh for an extended period of time and is the subject of a research paper, published in 2014. The researchers look at how Pittsburgh Public Schools utilizes a three-pronged approach to gauge a teacher’s effectiveness. These three domains of evaluation (professional practice, student surveys, and value-added measures) are positively correlated, which suggests, according to the researchers, that “they are valid and complementary measures of teacher effectiveness” (Chaplin, Gill, Thompkins, & Miller, 2014). The first component of the observation and evaluation regimen that the Pittsburgh Public Schools requires is a professional practice measure that is based on the Charlotte Danielson Framework for Teaching, known as the Research-based Inclusive System of Evaluation (RISE), which relies on principal evaluations of teachers. This is coupled with a student survey that seeks to “incorporate students’ perceptions of teachers’ practices and was developed by Ronald Ferguson of Harvard University as part of the Tripod Project and administered by Cambridge Education” (Bill & Melinda Gates Foundation, 2012). The third component of evaluation is based on a value-added measure that charts student changes on test scores over the three previous years’ worth of teaching (Chaplin et al., 2014). This multi-domain approach allows school leaders and administrators to develop a more well-rounded approach to teacher observation and evaluation that could not be obtained by merely looking at any one of these three
  • 37. 25 areas independently. The researchers suggest, through their findings, that the comparability and positive correlation of the three types of evaluations strengthens the fact that they should be used to complement one another, not supplant other measures of effectiveness. “Although none of the measures represent a gold-standard benchmark, the correlations across them suggest that they are capturing various aspects of effective teaching in complementary ways” (Chaplin et al., 2014, p. iii). It is the use of a modified version of the Danielson Framework that warrants continued analysis and discussion. Thomas Viviano, in a 2012 research paper published in the Journal of Career and Technical Education titled “Charlotte Danielson or National Board Certification: A Comparison and Contrasting of Two Major National Frameworks for Teaching,” seeks to differentiate between the two major national models of teacher evaluation, identified in his quasi-eponymous title. The Danielson Framework includes four domains of professional operation (planning and preparation, the classroom environment, professional responsibilities, and instruction) that are divided into twenty three sub-domains that provide additional areas for observation and evaluation. Teachers operating in this framework are classified—within each domain or sub- domain—as one of “four levels of competency to include distinguished, proficient, needs improvement or progressing, and unsatisfactory” (Viviano, 2012, p. 114). The National Board for Professional Teaching Standards (NBPTS) measures “five core propositions on what teachers should know and be able to do” and within these five areas there “are a total of sixteen subject matter areas that a teacher can earn a certificate in within various developmental levels for a total of twenty five certificates in all” (p. 115). These Danielson domains are compared with their most closely related NBPTS propositional area in the figure below, Figure 1, “looks at the comparisons of the two national frameworks and how they cross reference.” (p. 116).
  • 38. 26 Figure 1 Danielson Framework for Teaching in comparison to the NBPTS’s Core Propositions Charlotte Danielson’s Four Domains NBPTS’s Core Propositions Planning and Preparation Teachers are committed to students and their learning Classroom Management Teachers know the subjects they teach and how to teach those subjects to students Instruction Responsible for monitoring and organizing student learning Professional Responsibilities Teachers think systematically about their practice and learn from their experience Teachers are members of learning communities This figure “looks at the comparisons of the two national frameworks and how they cross reference. NBPTS has two categories that engulf professional responsibilities and two for instruction” (Viviano, 2012, p. 116). Between the Danielson Framework and the NBPTS, there are two minor differences that Viviano identifies; these differences relate to “using data to help guide and plan curriculum and teaching methods, and showing professionalism. Danielson’s framework addresses both of these categories and NBPTS addresses neither” (2012, p. 118). In the national standards for teaching excellence outlined in the legislation and policy surrounding the George W. Bush-era No Child Left Behind Act, “data collection to improve teaching and learning has become very important
  • 39. 27 and is not only the responsibility of administration but also now the responsibility of teachers,” and the presence of this measurement within the Danielson Framework lends credence to this model’s inclusivity of contemporary issues in educational instruction and supervision (p. 118). Apart from the non-inclusion of data in the NBPTS, he also discusses the Danielson Framework’s incorporation of professionalism as a matter of professional practice, further stating that it is “an essential component if you are going to use a framework for assessment and evaluation” because it is important for educators to “look at the comprehensive teaching professional and professionalism as crucial in that so many young lives are dependent on role models to pave the way toward strong and responsible citizenship” (p. 118). Viviano, in his conclusions, brings up important points that relate to how the Danielson Framework and the NBPTS should be used in evaluations and assessments, bringing up the important observation that any time a principal or supervisor “evaluates a teacher, he or she is placing a worth on another human being’s skills” (p. 118). It is important, he conveys, that after an assessment, “the administrator and teacher should concern themselves with what can be done to ameliorate any problems or skill deficiency that was revealed during the assessment process in order to benefit students.” It is from here, with the understanding that comes from utilizing a professionally developed standard of evaluation such as the Danielson Framework, that the administrator then “merely becomes the facilitator to make sure that the teacher goes through the right professional development needed such as a workshop, mentoring from a fellow teacher, a coach, the administrator, or research” (p. 118). It is this intricate professional relationship shared between teachers and their supervisors that is the subject of Callahan and Sadeghi’s research, published in 2015, that looks at teacher perceptions of evaluation models in New Jersey’s public schools. In 2012, the state legislature of
  • 40. 28 the State of New Jersey—in response to motivation from the Obama administration’s Race to the Top program—signed into law a program that “calls for a four level evaluation system of teachers that links individual student data to teachers and creates a more difficult process for teachers to earn tenure” (p. 47). Boldly, the law also “targets teachers who have already earned tenure” and, “in a major change to educational policy, tenured teachers may lose their jobs after two consecutive years of ineffective evaluations” (p. 47). In order to gain a greater understanding about this valued-added coupled with principal evaluation approach, the researched conducted a survey of more than six hundred eighteen public school teachers across the state. The findings of this survey prove illuminating. Callahan and Sadeghi identified that the state-required minimum observation requirements did increase the quality and duration of classroom evaluations by principals: in 2012, the first year of implementation, “formal evaluations were conducted infrequently with a varying degree of accuracy and impact” and “nearly half of the teachers indicated the formal evaluations did not lead to improvements in their practice” (p. 56). But in a follow up survey conducted in 2014, two academic years after the program’s start, teachers did identify an increase in the rate of observation, but concerns were raised about the diminishing value of the observations themselves. A formulaic observation regimen, they argued, led teachers to be more concerned with the technical process of observation as compared to its supposed end goal: improving classroom instruction. Several teachers “noted that their principals were more focused on entering observations in real time then on teacher-centered observations. They appeared more focused on entering information on tablets, then in actually observing.” Teachers further noted that “the technology and demands of observing numerous required elements made the observation scripted” and, therefore, unproductive (p. 56).
  • 41. 29 In their conclusion, the authors anchor their findings to the programmatic intentions of the teacher evaluation and tenure system, known as ACHIEVE NJ, identifying that the process of required minimum observation standards by principals has transformed, with detrimental effects, what was once “an organic, albeit infrequent, process” into a scripted one. This shift has produced generally demoralized educators, of which one contributing factor is the state- mandated emphasis on rating teachers (p. 56). The writers’ findings are summarized in the final paragraph of this particular piece of peer-reviewed literature, remarking that: teacher evaluation systems are not perfect and effective teachers are not the product of formulas. Research shows us that much of what effective teachers do cannot be measured by categorical ratings. However, that is not to say we should not attempt to define what effective teachers do and make every effort to replicate it. We need to move beyond checklists and rubrics that fail to acknowledge teaching excellence and we need to identify and offer professional development strategies that are most effective in improving teaching pedagogy and ultimately improving student achievement (p. 56). But if teachers are demoralized from being categorically ranked and evaluated, then not provided adequate or appropriate professional development, what does a working model look like? Shaha, Glassett, and Copas seek to answer this question with a twenty seven state study that aims to identify the “impact of teacher observations in alignment with professional development on teacher efficacy,” which was quantified by analyzing two hundred ninety two schools, operating in one hundred ten districts and more than two dozen states. These three authors argue that the end goal of observation and evaluation within a school setting is to provide the proper environment and interactions that “result in improved student
  • 42. 30 capabilities and achievements” (2015, p. 55). However, a significant amount of research has shown that this relationship is strained and that schools chronically find the most appropriate manner to “identify ways to continuously improve the impact teachers have on their students” (p. 55; Duke & Stiggins, 1990; U.S. Department of Education, 2013). The most common approach to pursue the continued enhancement of instruction is professional development (Buczynski & Hansen, 2010). A cannon of documentation and research bears light on the truth of the effectiveness of professional development, however, which does not necessarily jive with the expected findings. In spite of increased attention and focus on the role of professional development in teacher improvement and support, “data substantiating improved impact of teachers on students remains sparse” (p. 55; Avalos, 2011; Desimone, Porter, Garet, Yoon, & Burman, 2002; Farnsworth, Shaha, Bahr, Lewis, & Benson, 2002; Garet, Porter, Desimone, Birman, & Yoon, 2001; Lewis et al., 2003; Shaha, Lewis, O’Donnel, & Brown, 2004). “Teacher observation is another technique widely used wherein school leaders, experts, or designated coaches watch teachers and then provide feedback or guidance aimed at improving impact for classrooms and student learning” (Shaha et al., 2015, p. 56). Many researchers since the turn of the new millennium have indicated, nevertheless, that traditional teacher evaluations fail to appropriately inform teachers of areas for improvement, rarely occur often enough to be beneficial, and have not proved consistently effective beyond simply being a necessary action to fulfill bureaucratic responsibilities (Darling-Hammond, 1986; Hazi & Arredondo Rucinski, 2009; Jacob & Lefgren, 2008; Peterson, 2000; Stiggins & Bridgeford, 1985; Weisberg et al., 2009). Consequently, Shaha, Glassett, and Copas all argue in their writing that supervisory observation as a means of teacher improvement provides only minimal or mixed outcomes that do not drive toward improved student achievement (p. 56). It is in the pursuit of identifying best
  • 43. 31 practices of evaluation and observation (followed by appropriate professional development) that these authors have conducted their research. One of the most central findings of Shaha, Glassett, and Copas’s research is that, when teacher observation is combined with appropriately matched professional development, student performance in class improves (2015, p. 58). Similar research-based conclusions point to a direct and positive correlation between the amount of time a teacher is observed in the classroom by their supervisor and greater gains in student accomplishments on standardized math and English assessments (p. 58). The recommendations based on this research point to the important relation between a principal’s observation of a teacher and the pertinent recommendation of professional development opportunities, whether in an on-demand, online program or through a more traditional approach. Consequently, when schools and districts and states “work to improve the education in their schools and in those throughout America and the broader educational world, the central focus of validation and verification efforts must include the evaluation of teachers for the purpose of improving their impact on students” (p. 60). And if schools seek to improve student academic outcomes, it is important that, “although observation-based teacher evaluations might be criticized and disconnected from the needs of students” research indicates that “a coordinated approach involving [professional development] recommendations and executions is impactful for student advances” (p. 60). A new trend in education research seeks to identify how this customary approach to observation (conducted solely by the principal or supervisor) can be supported and supplemented by peer observation of teaching (Eri, 2014). In order to contextualize peer observation of teaching, Eri, in his 2014 publication Peer Observation of Teaching, provides a definition for the practice as a “reciprocal process where a peer observes another’s teaching” and then “provides
  • 44. 32 constructive feedbacks that would enable teaching professional development through the mirror of critical reflection by both the observer and the observed” (p. 625; Brookfield, 1995). Eri identifies five primary benefits of peer observation of teaching which include: (1) enhancing the quality and skills of the observed teacher, (2) participants gain confidence in their teaching methods, (3) new ideas are acquired for more effective instruction, (4) teaching methods and resources can be shared more easily, and (5) it creates a mutually shared assurance of “continued commitment to teaching” (Eri, 2015, p. 625). The author further identifies the procedure that peer observation of teaching should follow in order to be most effective. The process begins with a pre-observation, where the observer and observee identify the milieu of the observation, including time and location, as well as the particular classroom practices to be observed. After this initial meeting comes the observation itself, which should be for a more than merely cursory amount of time and include detailed note taking and critical observation. This is followed by the post-observation meeting where feedback is provided and a conversation is held that explores the nature of the observation and the thoughts of both participants. Subsequently, both individuals engage in a personal process of critical reflection, a semi-metacognitive effort, to truly think about the teaching and observation and discern the value that has been borne of it. The final stage is to implement any suggestions for change and improvement (p. 626–629). This background on peer observation of teaching leads to a more in-depth discussion on the topic of providing appropriate feedback to practicing teachers, whether this be from colleagues or supervisors. In the 2011 research study “Teacher Conferencing and Feedback: Necessary but Missing,” published in the International Journal of Educational Leadership Preparation, Anast-May, Penick, Schroyer, and Howell seek to address a common concern. The authors note that because “there is an absence of systematic feedback for teachers to facilitate
  • 45. 33 [the] professional growth [of teachers and to] improve instruction,” evaluators “tend not to provide detailed and concrete feedback after they have observed teachers” and that “without objective feedback and regular reports on progress and performance, an individual is less likely to achieve his or her professional goals” (p. 2). On the other hand, “constructive and meaningful feedback is needed to promote reflection and allow teachers to plan and achieve new goals, which will ultimately lead to an increased sense of efficacy in their teaching” (p. 2). The researchers have found that a significant portion of supervisory feedback tends to be shallow, unhelpful, and inaccurate, rarely verging on helpful or encouraging (p. 2; Frase, 1992). At the end of the day, it is argued, “an evaluation has no meaning if it is not interpreted, questioned, discussed, and reflected on, ultimately leading to making different and more effective decisions” (p. 2; Feeney, 2007). Turnbull, Haslam, Arcaira, Riley, Sinclair, and Coleman (2009) identified that one of the chronic shortfalls in the evaluation of teachers is that there is a disconnect between the amount of time a teacher is observed and the amount of feedback received. These researchers also found it common that “principals provided no individual feedback, choosing instead to focus on group feedback based on a checklist criteria” (Turnbull et al., 2009). Two other researchers have found that this is not merely an issue affecting the most inexperienced teachers, but that meaningful feedback for teachers outside their probationary periods was almost non-existent (Kelley & Maslow, 2005). This knowledge is what inspired Anast-May, Penick, Schroyer, and Howell to conduct research that sought some answers as to why teachers tended to be skeptical about increased observation, then provide recommendations for professional practice. Although their research was on a more limited scale (three elementary schools with a combined total of two thousand two hundred twenty five students), the research team was able to
  • 46. 34 safely conclude that “the process of evaluation should involve conferencing and feedback that will lead teachers to construct their own understandings and set professional goals that are measured in terms of student learning” (p. 6; Wheatley, 2005). Similarly, the researchers support the notion that data collection should play a role in the effective evaluation of teachers, stating that “measurement [should] be used from a deeper place of understanding, the understanding that the real capacity of an organization arises when colleagues willingly struggle together in common work that they find meaningful” and, additionally, that measurement provides “the kind and quality of feedback that supports and welcomes people to step forward with their desire to contribute, to learn, and to achieve” (p. 6). But the incorporation of data into teacher evaluation does not solely come from the analysis of student test scores on standardized assessments. Student surveys are becoming more prevalent in determining the effectiveness of particular teachers (Liu et al., 2014). Liu and her colleagues seek to answer a poignant question through research: “Does adding teacher and student survey measures to existing measures (such as supervisor ratings and student attendance rates) increase the power of principal evaluation models to explain across-school variance in value-added achievement gains?” (para. 1). The findings are starkly evident and reassuring. Utilizing a two-step multivariate regression analysis, it was determined that incorporating “teacher and student survey measures on school conditions to the principal evaluation model can strengthen the relationship between principals’ evaluation results and their schools’ average value-added achievement gains in math and in a composite of math and reading” (p. i). A significant number of states, spurred on by their legislatures and state departments of education, have begun to seek out multi-variable measures for teacher effectiveness. Many
  • 47. 35 districts have shown a keen interest in incorporating teacher evaluations conducted by students (Illinois State Board of Education, 2011; Mattson Almanzán, Sanders, & Kearney, 2011; National Conference of State Legislatures, 2011; Wacyk, Reeves, McNeill, & Zimmer, 2011). This drive to incorporate student surveys follows a recognition of the need to bring in more than simply one evaluator’s observations as the basis for determining an effective teacher. New models being developed rely on multiple performance measures such as growth in student achievement, leadership competency assessments, and school climate surveys. This creates a more complete picture of principal effectiveness (Clifford, Behrstock-Sherratt, & Fetters, 2012; Illinois Principals Association & Association of School Administrators, 2012; Mattson Almanzán, Sanders, & Kearney, 2011; The New Teacher Project, 2012; Ohio Department of Education, 2011; Roeber, 2011; Wisconsin Educator Effectiveness Design Team, 2011). In the research conducted by Anast-May, Penick, Schroyer, and Howell, new information is contributed to the discussion that schools and districts are currently having about whether to include student surveys about teachers in the supervisory evaluation of teacher effectiveness. The answers to this contemporary question were discerned by studying one particular school district within the American Midwest region in depth during the 2011–2012 school year, encompassing more than thirty nine total schools (2015, p. 2). The research sought to determine if student surveys of teachers correlated in any way to standardized test scores in English, math, and science, utilizing the Measure of Academic Progress (developed and administered by the Northwest Evaluation Association) (p. 2). The student survey used was the Tripod Student Perception Survey, which was developed by Harvard researcher Ronald Ferguson. This survey “consists of thirty-six items measuring students’ perceptions of their classroom instructional environment in seven domains and six items measuring students’ perceptions of school safety
  • 48. 36 and climate” (p. 3). Through the administration of this survey, it was confirmed—as numerous previous studies have also shown—that the quality of the classroom teacher significantly impacts student achievement (p. 6; Aaronson, Barrow, & Sander, 2007; Gordon, Kane, & Staiger, 2006; Kane & Cantrell, 2010; Kane, Rockoff, & Staiger, 2006; Rivkin, Hanushek, & Kain, 2005). What Anast-May et al. have been able to show is that this survey accurately predicts value-added measures of teacher and school academic success on standardized tests, a novel finding (p. 7). One of the primary supporting findings is the importance of school-level leadership on student outcomes (p. A1–A2). In supporting literature, especially in a meta-analysis of more than twenty existing research papers, it was determined that the “average effects of instructional leadership practices on student achievement and other outcomes (such as absenteeism and engagement) was three to four times as large as the average effects of other leadership practices that do not explicitly focus on curriculum and instruction” (Robinson, Lloyd, & Rowe, 2008). By measuring student perceptions of school leadership (which the aforementioned Tripod Survey includes) it is possible to draw a direct connection between the efficacies of a school’s principal and the outcomes that students are expected to achieve. One state that has begun the process of implementing a multi-domain teacher effectiveness evaluation program that includes both value-added measures as well as traditional observation coupled with student survey responses is Arizona (Lazarev, Newman, & Sharp, 2014). This program of teacher evaluation began as a pilot project in five varying school districts (four public and a charter network) throughout the state (p. i). Arizona chose to change its teacher evaluation models in the wake of changes to the Elementary and Secondary Education, Act, when states were required to submit various plans to the United States Department of Education including their plans on how teacher evaluations needed to change to support a more
  • 49. 37 rigorous focus on instructional outcomes (U.S. Department of Education, 2012). Arizona was one of the “two-thirds of U.S. states [that] have made changes to their teacher evaluation policies since 2009” (Jerald, 2012). Significant bodies of evidence have been compiled which have “yielded empirical evidence of correlations between various teacher effectiveness metrics, including scores from several widely used classroom observation instruments, student surveys, and estimates of teachers’ value-added contributions to student test achievement” (Lazarev et al., 2014, p. 1; Kane & Staiger, 2012). States, however, are often left with little existing research about how best to integrate these varying evaluation tools in a meaningful way and to analyze and utilize collected data (Rothstein & Mathis, 2013). It is to provide this empirical background for implementation that Lazarev, Newman, and Sharp began to explore how this multi-domain evaluation system was being implemented in the Grand Canyon State (p. 1). The State of Arizona’s three domains of teacher evaluation include: (1) teacher observation, (2) student academic progress, and (3) surveys (p. 2). Arizona chose to base their state’s observation instrument on the Danielson Framework (Danielson Group, 2011). This study chose to focus its effort on determining the relationship between the observation of teachers by principals and the other two data collection instruments. And for good reason—one of the most startling findings of the report is that the collected “results suggest that the observation instrument was not used in a manner that effectively differentiated among levels of teacher practice” (Lazarev et al., 2014, p. 5). Consequently, “there is evidence that observations by school principals tend to produce inflated scores—in particular, many teachers with students [who demonstrate] low academic progress receive high observation scores” (p. 5). Despite the tendency of principals to over-praise a teacher’s effectiveness in the classroom, the researchers
  • 50. 38 were able to draw lines of positive correlation between the scores that a principal provided utilizing the Danielson Framework and student academic achievement measures and surveys. This means that the Arizona teaching framework (a replica of that produced by Charlotte Danielson), consisting of twenty-two sub-domains for evaluation, can prove to be a statistically validated measurement tool and, “if the observation items measure a single underlying aspect of teaching effectiveness, a single aggregated observation score obtained by summing or averaging item scores would be a valid measure of teaching” (p. 9). Of the four domains of the Danielson Framework (planning and preparation, the classroom environment, professional responsibilities, and instruction), the two areas that were most highly associated with increased student achievement were the two that must be assessed outside the classroom: planning and preparation and professional responsibilities (p. 17). Lazarev and his colleague-researchers, then, cautioned any districts seeking to implement a multi-domain evaluation tool to make sure that principals are well trained and know how to accurately evaluate a teacher utilizing the most empirically-sound observation models (U.S. Government Accountability Office, 2013). But the opportunity exists for strong adoption in other states. The literature and history of teacher evaluation points to a continual endeavor to find the most appropriate manner by which to evaluate an educator’s craft and effectiveness in the classroom. It is a task that must be done, regardless of its challenges, because a failure to appropriately evaluate a teacher is a failure to control and monitor and cultivate the best learning environment possible for students in educational settings. As the MET Project (Measures of Effective Teaching), funded by the Bill & Melinda Gates Foundation, succinctly points out: “effective teaching is teaching that leads to student success” (MET Project, n.d.). In order to do this most effectively, today’s educational researchers have begun to place a heavy emphasis on
  • 51. 39 finding reliable and valid measures of teacher effectiveness that strongly correlate to student academic success; after all, “teaching evaluations are only valuable if they can be counted on to accurately reflect what a teacher is doing in his or her classroom that is helping students succeed” (MET Project, n.d.). Because teaching is recognized as a multi-faceted discipline, it becomes evident that no single measure of success can accurately portray how effective a teacher is performing in the classroom; for example, teaching takes significant amounts of planning and preparation, and while circumstantial evidence might be present when a principal observes a classroom, it can only be truly measured by other means. The Implementation of Evaluation It was the fourth century B.C. Greek philosopher Aristotle who, as quoted by Diogenes Laertius in Lives of the Philosophers, stated that “the roots of education are bitter, but the fruit is sweet” (Laertius, 1925). While the history of evaluation illustrates the manifold transformations that supervisory observation has undergone, and the struggles between competing ideas and ambitions, one end goal has remained: that a teacher evaluation must, in its final calculations, lead to the most efficacious of teachers providing a powerful and transformative and academically rich educational environment for students. But the path toward this final realization is often fraught with the question of practical implementation, this point being illustrated with the case of a disappeared Los Angeles teacher in 2010. Rigoberto Ruela’s body was eventually found in the wooded slopes of Big Tujunga Canyon (Zavis & Barboza, 2010). Twenty-six miles from the school he loved so dearly, where he had taught fifth grade for fourteen years, the Angeles National Forest’s overgrown redwoods could only hide Ruela’s body from the search-and-rescue team for so long before emancipating
  • 52. 40 its martyred educator. But this was not a hiking accident: the Los Angeles County Coroner quickly ruled that the cause of death was suicide (Strauss, 2010). The search-and-rescue team was only dispatched when Mr. Ruela failed to show up for work at Miramonte Elementary School, a public school located in south Los Angeles, tucked inside a community riddled with gang-violence and aching poverty (Zavis & Barboza, 2010). His coworkers knew how dedicated Ruela was; the school’s administration confirmed that it was a rare occasion that he did not show up to school—his record of attendance was nearly perfect over his more than decade tenure—and never unannounced (Lovett, 2010). His colleagues, however, could tell that he was slipping into a depression in the weeks before his absence and death. In spite of the time that Mr. Ruela put into tutoring students after school or communicating with parents (or even the complimentary review that he received from his supervisor during the previous 2009/2010 school year), his family blames one particular event for pushing him into a tailspin that ended with a base-jump from a bridge: a poor teacher review and ranking, as calculated in a value-added manner by the Los Angeles Times and published publically online, which rated him as a teacher who was “less effective than average” (Romero, 2010; Lovett, 2010). Rigoberto’s brother Alejandro, in an interview with a Los Angeles public radio station, announced that he does “blame the [Los Angeles] Times [for his brother’s death] for the fact that they did publicize [his scores and ranking]… When you have the L.A. Times labeling him that way that really hits you hard. He took his job seriously” (Romero, 2010). This story’s beginning lies, however, back in the year 2010 when the Los Angeles Times, the major metropolitan daily newspaper in Southern California, posed a bold question: “Who’s teaching L.A.’s kids?” (Felch, Song, & Smith, 2010). Working with the Los Angeles Unified
  • 53. 41 School District, the newspaper evaluated the previous seven years of teachers’ impacts on student test scores. (Abramson, 2010). These scores, designed to measure the effectives of teacher instruction by a value-added measure, were supposed to be a non-biased, mathematical means by which to judge the worth of a classroom instructor (Hancock, 2011). Despite warnings and concern from researchers and leaders in education that the findings provided by the Los Angeles Times were unreliable, the newspaper decided that it would publish the data, and this was hailed as a monumental step by Arne Duncan, the then-U.S. Secretary of Education, who encouraged other newspapers to follow the Times’s example (Anderson 2011; Felch & Song, 2010). “What’s there to hide?” Duncan is quoted as saying the day after the Times first published the teacher evaluation; “in education, we’ve been scared to talk about success” (Freedberg, 2011). This message from the administration of the Barack Obama mirrored the federal government’s push for localized value-added measurement of teacher effectiveness, as spurred on by “Race to the Top.” This federal program sought to financially encourage states to make it a legal requirement that teacher evaluations include, as their basis, quantifiable data on how well teachers were improving student scores on standardized tests. It is a seemingly logical evaluation, its developers argued, as any district should be willing to make a student’s growth in knowledge a central component of determining a teacher’s value to a particular district and the students he or she teaches (Strauss, 2012). The Association of Supervision and Curriculum Development (a not-for-profit organization of principals, superintendents, teachers, and others in the field of education) defines value-added assessment simply as measures that capture “how much students learn during the school year, thereby putting teachers on a more level playing field as they aim for tenure or additional pay” (David, 2010).
  • 54. 42 As the debate in Los Angeles heated up in the aftermath of Mr. Ruela’s death, the head of the union of Los Angeles’s public school teachers said that these types of “value-added assessments are a flawed system.” The Los Angeles Times subsequently released a statement of its own, declaring both its sympathy for the death of the teacher, but also that The Times published the database, which is based on seven years of state test scores in the L.A.U.S.D. schools, because it bears directly on the performance of public employees who provide an important service, and in the belief that parents and the public have a right to judge the data for themselves (Lovett, 2010). These two camps, on opposing sides of the debate, illuminate a larger trend in the evaluation of teachers and schools, and the debate about which particular factors best fit into the calculus of determining how effective a particular educator is in the classroom.
  • 55. 43 CHAPTER 3: METHODOLOGY This school-community action project seeks to indigenize the Danielson Framework for application at Little Wound School. This will be accomplished by surveying students, teachers, and administrators from Little Wound School to identify internal beliefs about which components of the Danielson Framework (adopted by South Dakota as the South Dakota Framework for Teaching) are the most closely associated with what makes for a most effective teacher. The collected information will be used to devise a uniform evaluation model for use when principals and supervisors observe classroom teachers. Subjects In order to evaluate the school’s perceptions of the importance of the various components that comprise the Danielson Framework for Teaching, it is important to identify the population of individuals who were surveyed for their opinions. Three primary groups were identified to take the survey: (1) students over eighteen years old, (2) teachers, and (3) administrators. Because only students who are over eighteen years old will be surveyed, these students will be in the twelfth grade. Only students at this age juncture will be surveyed to eliminate the participation of vulnerable populations in the study, which includes youth below eighteen years of age. While this limits the amount of student input, the presence of any of this population’s input will still provide an opportunity for analysis. This group includes both male students (8) and female students (19). All students surveyed were Native American (based on the designation on the survey) and attended school between February 22 and February 26, 2016. All teachers within the school between kindergarten and twelfth grade were provided a survey; interventionist teachers were also included, as were teachers of specials in the elementary school. (These two latter groups were included because they are subject to the same model of
  • 56. 44 evaluation by their direct supervisors.) The gender breakdown of teacher research participants were male (19) and female (34). A majority of the teachers surveyed were Native American (71%; 63 out of 88). The average tenure of teaching experience for a teacher-respondent was 10.8 years (median = 6), with some teachers having no prior teaching experience (zero years of experience), while the most experienced teacher has taught a total of 34 years. Teachers were surveyed the same week as the students. Administrators were also surveyed. This group was the smallest of those surveyed, with a total of eight individuals. This group included three principals (elementary, middle, and high school), the superintendent, and other high level individuals. Of this group, seven are female and one is male; all identify as Native American. The following table, Table 1, identifies the rates of response from broken-out groups who were surveyed for this research. (Information for the section of the table labeled “sample pool size” is current as of the Friday of the week the survey was taken; all teachers and administrators within the total pool of available survey-takers were provided with a survey.) TABLE 1. Rate of survey responses. Sample pool size: Returned surveys: Percentage completed: Students Over eighteen years of age 28 27 96% Total: 28 27 96% Teachers Elementary (K–5) 20 18 90% Middle school (6–8) 11 11 100% High school (9–12) 24 24 100% Total: 55 53 96.4% Administrators Total: 8 8 100%
  • 57. 45 In order to ensure privacy, no identifying information was collected from survey respondents. Because of the sample size, it is impossible to track back one survey to the student who took that particular survey. The privacy of teachers has also been considered, and no identifying information appears within this report or on the surveys distributed to teachers. Similar precautions apply to the administrators. In order to confirm that the privacy and additional rights of all subjects were protected, the research proposal for this report was submitted and approved by relevant area actors, including the Oglala Sioux Tribe’s Research and Review Board, the Oglala Lakota College Institutional Review Board, and the Little Wound School Board and superintendent. Procedures and Data Collection The Little Wound School Board approved this research project to be conducted through the school itself—the research collected and the development of a model teacher evaluation in compliance with state and BIE requirements will directly benefit the school and its teacher observation and evaluation needs. Consequently, all teachers in the school will be provided with a survey asking them to identify, within each of the four domains of the Danielson Framework, the one component that makes for a most effective teacher. Teachers will also be asked for their gender, their total years of teaching experience, whether they are an enrolled member of a federally recognized tribe, and if they have previously had any formal training on the Danielson Framework. In total, there will be twenty surveys distributed to elementary school teachers—this includes all classroom instructors, two Lakota language instructors, one physical education teacher, and two intervention teachers: one English, one math. Within the elementary school, only these teachers
  • 58. 46 will be surveyed because it is only these staff members who are formally evaluated by their direct administrators in compliance with state and Bureau expectations. In the middle school, eleven teaching staff will be surveyed; this includes eight core- subject teachers, two Lakota language instructors, and the instructor of the middle school’s alternative education program. In the high school, the survey will be distributed to a total of twenty classroom teachers. Like the elementary school, these staff members are formally evaluated by their principal supervisors, and thus will be surveyed for their input about the most central domain components to be utilized in a model teacher evaluation form. A total of eight school administrators will also be provided with a survey. These administrators will be asked to identify their number of years of administrative experience, their total years of teaching experience, if any, and whether the survey-taker has had any training provided by the originators of the Charlotte Danielson Framework for Teaching. For all teachers and administrators, the survey—with an accompanying letter explaining the purpose of the survey, the fact that it is voluntary, and that no identifying information will be collected—will be placed in their individual mailboxes on a Monday morning before school starts. (All teachers and administrators will be informed the week before by email that the survey will be handed out the following week.) Teachers and administrators will fill out the survey independently and submit the document to a sealed box located near all participants’ respective mailbox clusters (one in the elementary school, one in the middle school, one in the high school, one in the administrative offices). Staff members will have until the Friday of that week to return their completed surveys for inclusion in the research project. In this manner, all returned surveys will remain entirely anonymous and the non-identifying information will be held confidential.
  • 59. 47 Because of the inclusion of only students above eighteen years of age, students will be selected in a non-randomized manner. These students will be provided the same survey as staff and administrators with only some minor changes for the information collected, and will include a letter of informed consent—like all prospective participants. These adult-aged students will be instructed to return their completed surveys to the same locked box for high school staff members. (These students will all be in the high school.) Because the survey will be conducted at a school operated under the auspices of the Bureau of Indian Education on a Native American reservation, it is an assumption of the researcher that most respondents will identify as enrolled members of a federally recognized Indian tribe. In order to confirm that this is, in fact, accurate, survey takers will be asked to identify if they are enrolled. While this question could have been rephrased to ask if a participant identifies as Native American, asking for confirmation of tribal enrollment is a method by which to gather this information in the same manner that the U.S. Census Bureau conducts its demographic research (Norris, Vines, & Hoeffel, 2012). Information collected is not limited to enrolled tribal members, and the responses from individuals who are not enrolled members will prove similarly enlightening when the data collected is analyzed. The participants are all informed that the survey is entirely voluntary in nature and the completion of the survey is not required. Similarly, in line with making sure that all the participants taking the survey are engaging in the activity with informed consent, the letter provided also points survey-takers to the lead researcher and research committee chairperson in case of discontent, and also includes information on contacting the Oglala Lakota College and tribal institutional review boards in case of concern. These letters of consent were attached to the front of all surveys distributed, and were returned with completed surveys. In a similar manner to