Dr. Bobbie Eddins, Dr. Jeffrey Kirk, Dr. Dorleen Hooten, Dr. Brenda Russell - Published in NFEAS JOURNAL, 31(1) 2013-2014 - Dr. William Allan Kritsonis, Editor-in-Chief, NATIONAL FORUM JOURNALS (Founded 1982) - www.nationalforum.com

NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL
VOLUME 31, NUMBER 1, 2013

UTILIZATION OF 360-DEGREE
FEEDBACK IN PROGRAM ASSESSMENT:
DATA SUPPORT FOR IMPROVEMENT
OF PRINCIPAL PREPARATION
Bobbie Eddins
Jeffrey Kirk
DorleenHooten
Brenda Russell
Texas A&M University–Central Texas
ABSTRACT
Crucial to each school leader’s success in the complex environment of a prek-12
campus is development of knowledge and skill in a relevant preparation
program. In collaboration with school district leaders and school leadership
practitioners, faculty members in the school of education at a regional
university have designed and implemented a learn-as-you-go self-assessment
approach for continuous improvement of the master’s degree/principal
certification program for prek-12 school leaders. Grounded by a focusing
mission and an action research case study approach, the approach utilizes 360degree feedback that provides multiple-source data linked to the state and
national principal standards. It borrows the circular concept from the widely
utilized individual 360-degree feedback assessment practice that provides both
depth and breadth concerning a leader’s effectiveness. Data from internal and
external 360-degree feedback sources include an ongoing review of school
leadership literature, a self-assessment by program faculty, a critical review by
educational leadership experts, an analysis of internal and external student
performance data, focused conversations with advisory groups, and perceptions
of program completers as well as their supervisors as they move forward on
professional leadership pathways. This feedback supports data-informed
decisions that strengthen key program components – student recruitment and
selection, program curriculum, instructional delivery including a rigorous
internship and mentoring support, stakeholder involvement, program staffing
and faculty development, and program planning and evaluation.

5

18NATIONAL FORUM OF EDUCATIONAL ADMINISTRATION AND SUPERVISION JOURNAL

Introduction

F

rom the moment school leaders are hired, they are expected to
mobilize the school community in improvement efforts focused
on success for all students. Leading schools today means
developing a culture of joint responsibility focused on a vision of
prosperity, opportunity, and creativity (Hargreaves & Shirley, 2009).
It requires growing student, teacher, staff, parent, and community
leaders who are engaged in shared inquiry and innovative responses
concerning learning challenges across an increasingly diverse
community of practice (Seashore Louis, Leithwood, Wahlstrom, &
Anderson, 2010; Sergiovanni, 2006; Glickman, 2003). Crucial to each
school leader’s success in this complex environment is development of
knowledge and skill through relevant preparation.
Principal preparation program quality is habitually criticized
by policy makers, school leaders, scholars, and professional
organizations. Concerns are raised about the relevancy of practice to
theory connections related to real time school purpose and capacity
building (Fullan, 2009; Orr, King, &LaPointe, 2010); the need for a
better
aligned
and
more
rigorous
curriculum
(Darling‐Hammond,LaPointe, Meyerson, & Orr, 2009; Lashway,
2003), the lack of substantial clinical experiences (Frye, Bottoms, &
O’Neill, 2005; Levine, 2005), and the underutilization of program
assessment (Orr, 2009). In short, for principal preparation program
completers to make an impact as school leaders, they must engage
inprogram learning that is relevant to prek-12 school environments,
delivered in a coherent and engaging sequence, facilitated by
knowledgeable and experienced faculty, and assessed through both
internal and external measures of effectiveness.

The Critical Role of Program Assessment
A growing emphasis on accountability for educator preparation
continues to increase motivation for ongoing assessment about the

EDDINS, KIRK, HOOTEN, & RUSSEL19

program and quality guarantees for program completers. Program
evaluative assessment–the process of systematically collecting and
analyzing data about a program to determine its significance and
improve its performance – provides valid and reliable information to
decision makers about program results. As with any assessment
process, data is collected that supplies answers about key program
components and outcomes. Inquiry concerning program improvement
includes questions such as: ―Does the program succeed in doing what
it was meant to do?‖ ―How effectively is the program functioning?‖
and ―What modifications are needed to meet program goals?‖
Meaningful assessment of school leadership preparation is
critical to continuous program improvement, particularly in view of
the constantly shifting requirements of certifying entities and the ever
changing knowledge and skill set associated with school leadership
roles.The importance of this type of assessment for principal
preparation programs rests in its usefulness as footing for change that
will improve student learning and completer success. Effective
programs continually assess themselves against their vision, mission,
and goals, leveraging improvement based on measurable data rather
than impulse or tradition. Program self assessment in partnership with
the program’s many stakeholders provides positioning that ―reflects a
broad consensus concerning the most efficacious approaches to
preparing school leaders‖ (Glasman, Cibulka,& Ashby, 2002, p. 283).
The results provide program faculty, institutions, and consumers with
information to inform decision-making and policy development.
Those closely involved with preparation programs depend on
assessment processes to answer a number of questions related to the
effectiveness of their programs. Program-specific focusing questions
may include: ―What added benefit or value do program participants
receive?‖ ―How well prepared are the program completers for entrylevel school leadership positions?‖ ―How do employers feel about
their school leaders who completed the program?‖ ―How is the
program perceived by participants, completers, the university, and the
field?‖ ―What impact do the completers have on the school campuses


they serve?‖(Texas Principals Leadership Initiative, 2002).The
answers to the questions come from a variety of sources in the form of
perception and performance data. In much the same way that a
leader’s performance is scrutinized by internal and external
stakeholders in the widely accepted practice of individual 360-degree
feedback (Bracken, Dalton, Jaki, McCauley, &Pollman, 1997), the
school leadership preparation program may be centered within a circle
of known and trusted internal and external feedback sources. When the
many perception and performance data sources impart a ―surround
sound‖ or 360-degree effect, the combined evidence offers a more
inclusive ―look‖ at program effectiveness (Hernez-Broome & Hughes,
n.d.). This feedback coupled with relational supports such as coaching
and action research offers a path forward for mindful program
improvement.

An Assessment Approach to Anchor Continuous Improvement
In collaboration with regional school district leaders and school
leadership practitioners, school leadership program faculty members in
the department of professional education and policy studies within a
regional university’s school of education have designed and
implemented a learn-as-you-go planning and assessment approach for
continuous improvement of the school leadership preparation program.
All improvement efforts are anchored by a focusing mission crafted
and utilized by program faculty and stakeholders:The School
Leadership Preparation Program prepares school leaders who are
capable of facilitating the creative work of school communities that
leads to student and school success in the midst of a constantly
changing environment(Department of Educational Leadership and
Policy Studies at Texas A&M University-Central Texas,n.d.).
The following program learning goals coupled with defining
sets of competencies track the state principal standards and mirror a
set of certification test framework: (1) improvement of a school
community including vision, culture, communication, collaboration,


and ethical leadership; (2) development, implementation, and
evaluation of a school’s instructional program which meets the needs
of all students and includes curriculum, instruction, faculty and staff
growth, and instructional problem solving and decision making; and
(3) assurance of a safe and effective learning environment including
resource use and physical and support system management (Texas
Education Agency, 2010). In addition, program curricula is organized
by a set of integrated strands of learning in crucial areas such as a
generative and systemic decision making processes, integrated system
management, ethical leadership processes, and social justice equity
that unfold and are practiced across courses activities, all organized in
aProgram Instructional Strands by Course Alignment Learning Matrix
tied to state and national school leadership standards as well as the
NCATE ELCC Program Standards. Unpacking program curricula
across courses ensures learning in all goal areas; however, the skillful
integration of key elements is critical to school leadership
effectiveness (Robinson, 2010; Tucker, Young, &Korschoreck, 2012).
The case study format is built around a set of seven key
components: student recruitment and selection, program curriculum,
instructional delivery including a rigorous internship and mentoring
support, stakeholder involvement, program staffing and faculty
development, and program planning and evaluation. Providing the
spine for a multi-year state principal preparation improvement process
(Texas Principal Leadership Initiative, 2002), the corresponding
continuums for the components are consistent with a more recently
developed list of common features from an in-depth study of eight
leadership development programs (Darling-Hammond, Meyerson,
LaPointe, & Orr, 2009). Based on findings from the study, special
emphasis wordinghas been added to the original continuums for
student support, practicum experiences, and the internship experience.
Initial ratings about approach, implementation, and outcomes for each
of the seven component continuums have given program faculty a
baseline for tracking improvement within the case study format.
Exemplar ―Approach‖ language with special emphasis wording in
brackets for each continuum is provided below (Texas Principals


Leadership Initiative, 2002).
Program Involvement of Stakeholders
Faculty view collaboration between internal and external
individuals and groups as critical to keeping the program aligned and
viable. Input from a large group of stakeholders—student, graduates,
practitioners, and other university faculty—is solicited on a regular
basis using multiple methods.Involvement in implementing the vision
of the program is integrated in all improvement undertakings. The
faculty is actively engaged in work with personnel from schools,
districts, and state agencies. These personnel serve on advisory groups
for the program.
Program Improvement Planning
Engagement in planning is proactive and focused on
maintaining the vitality of the preparation program. Complacency is
low and the sense of urgency is high—a persistent discomfort with the
status quo characterizes the faculty’s approach to improvement
planning. A clearly defined continuous improvement process is used
to introduce and test proposed program changes. Alignment with the
faculty’s shared purpose and vision for the program is the litmus test
for any proposed change. Program improvement is a collaborative
rather than individual undertaking. Multiple types and sources of data
[tied to a set of goals linked to state and national school leadership
standards] are used to guide program improvement decision making.
Program Content
The program curriculum is a set of carefully sequenced
learning experiences focused on the principal’s primary role in
providing leadership for the improvement of teaching and learning in
elementary and secondary schools. Integrated into the program content
is a continuum of knowledge and skills that becomes more
sophisticated as the program progresses. Technical knowledge is


examined in terms of its relationship to the teaching and learning
process (e.g., instead of learning a school finance formula, the focus is
on learning how to budget allocated resources to achieve the school’s
improvement goals). Course offerings vary in format and credit
designation.
Program Delivery
The faculty [in consultation with program stakeholders]
decides on the types of learning experiences needed by the students as
they progress through the program. Accordingly, faculty members,
individually or as teams, introduce and use a wide variety of
instructional strategies (problem-based learning, action learning, real
time practicum activities) and media. Students are members of cohort
groups that are considered to function as learning communities.
[Expert mentoring and peer coaching are utilized.] Performance is
assessed using departmental rubrics that are published and which
include a variety of perspectives: self, peer, and faculty. Group
performance is also assessed. Students receive regular and ongoing
feedback about their performance [on both coursework and in an
intensive internship anchored by standards].
Program Staffing and Faculty Development
The university’s strategic plan gives priority to the recruitment
and hiring of a core of fulltime faculty with contemporary professional
experience. A proactive approach is used to identify and recruit highcaliber fulltime and part-time faculty. Part-time faculty is used to provide
expertise not possessed by the full-time faculty and are considered a key
part of the faculty. Fulltime faculty teach the same number of courses or
hours per week as their peers in other graduate programs, have equivalent
advising loads, receive equivalent compensation, and are rewarded
equally for the quality of their teaching and their engagement in
scholarship. The need for faculty to be engaged in the field with
practitioners is…valued [and] workloads are adjusted to accommodate
work in the field. Resources are allocated to address development needs


of the entire program faculty.
Student Recruitment and Selection
The faculty has developed a profile or portrait of the successful
student. There is written recruitment strategy that is shared with
schools and districts for feedback and to launch aggressive recruitment
initiatives. The goal is to recruit a diverse, capable, high potential
student body. Criteria for student selection are published and used
systematically during the selection process. Admission decisions are
made by the faculty upon review of the admission documents. There
is conscious control over the number of students admitted.
Commitment to a comprehensive set of measures is consistent with the
demands and expectations of the program.
Program Evaluation
All aspects of the program are rigorously evaluated in an
ongoing effort to keep the program aligned with the needs of the field
and best practices in leadership preparation. Feedback from those with
an interest in the program is sought on a regular basis. There is a
systematic plan for assessing the objectives of the program, the quality
of instruction and instructors, the nature of the curriculum and its fit
with student needs. The adopted evaluation plan calls for the use of
multiple forms of data in addition to feedback from students and
practitioners.

360-Degree Feedback Sources and Measurements
Grounded by the focusing mission and the case study
components, the continuous improvement approach utilizes 360degree feedback that provides multiple-source, multiple tool data
linked to the state and national (ISLLC) school leadership standards as
well as NCATE ELCC program standards.Some assessment data such
as performance on course practicum activities and final assessments,


student course evaluations, internship performance, and the master’s
comprehensive examination provide more of an internal data picture.
External ―reads‖ include performance on the state certification
examination as well as perception data fromprogram completer and
school district satisfaction surveys,providing rich data about program
effectiveness.This feedback supports data-informed decisions that
strengthen the seven key program components.
Student Perceptions and Performances
Feedback from students ensures an abundance of data about
their knowledge, skill and satisfaction in relation to program learning.
Assessments to measure student learning are embedded in every
course, measuring movement towards mastery of course learning
objectives through acommon set of program Grading Criteria and
Rubrics. Program rubrics are used for collaborative participation,
written products, course presentations, and professional and school
portfolio development.
Assessments for learning such as case studies, action learning,
problem-based learning projects, and action research are built into
course assessment plans. Many course assessments bear the label of
―practicum activity‖ because of the application to issues and
opportunities in students’ professional settings.Assessment of learning
measurements include student performance on the following: course
final assessments covering all course and learning objectives,
integrated practicum projects, action planning and performance during
the internship experience, and performance on the master’s degree
comprehensive examination and the state principal certification
examination.
The use of student course evaluations provides a tool to
measure student perceptions concerning the relevance of content and
the effectiveness of delivery. Students are also surveyed as they exit
the program upon completion and again when they serve in school
leadership roles. Completer data is organized by role and years beyond


completion.
School District Perceptions about Completer Performance
Of increasing importance to the continuous improvement work
by principal certificate program faculty are the perceptions and
recommendations of school district, community, and state
stakeholders. Principal mentors who supervise students’ internship
experiences and coach program alumni in school leadership positions
are surveyed concerning their perceptions of student and alumni
performance. Performance feedback for the university’s education
preparation programs are included in the state’s Accountability System
for Educator Preparation Program (ASEP). Information included in
this system along comprehensive needs assessment data provided
annually by the region’s districts inform program policy and practice
recommendations advanced twice a year by the Strategic Partners
Education Advisory Council (SPEAC) – a 43-member stakeholder
council that ensures a range of perspectives as it works to fulfill its
commitment to provide recommendations for continuous improvement
of all educator preparation programs housed at the university.
Early in planning efforts, SPEAC organizers determined that
an organized practitioner voice for each educator certification area was
needed to inform the council’s work. Certificate Area Practitioner
Sub-Councils (CAPS) have now been established to examine research,
best practice, and current program data, with recommendations for the
certificate areas then routed through the larger council. Thus, the
Principal CAPS advisory group has been convened as a 24-member
sub-council, with participation by some of the region’s most effective
elementary, middle, and high school principals. They represent a
diverse mix of campuses by size, location, and district affiliations in
the region. They provide perception data about alumni performance in
their districts, review data collected from other feedback sources, and
make recommendations for mindful program improvement.


Experts from the Field of Educational Leadership
Student and school district feedback is bolstered by a periodic
external Critical Friends Review which provides feedback to the
faculty engaged in work of program improvement. It is not a ―gotcha‖
process; rather, it is a comprehensive examination of current program
practice by respected colleagues engaged in the field of educational
leadership in programs in other states. The feedback clusters around
the seven program quality components, providing suggestions that will
leverage improvement defined by the component continuums.

Program Improvement Highlights
Faculty members use data from all of the measurement tools to
strengthen course content and instructional delivery. The Course
Design Reflections process supplies each course instructor with a
protocol to synthesize multiple sources of data and make decisions
about needed improvements. Each instructor is responsible for any
needed content revision, resource update, and activity and assessment
adjustment. Fulltime and adjunct program faculty meet twice a year to
check for alignment issues and depth of content coverage related to the
state principal standards, the domains and competencies of the
principal certification examination framework, the 2008 leadership
standards from the Interstate School Leaders Licensure Consortium,
and the leadership preparation program standards from
NCATE/Educational Leadership Curriculum Consortium. TheProgram
Instructional Strands by Course Alignment Learning Matrix is used to
anchor the alignment discussion.
Course reflection and collaborative alignment efforts have
yielded some very useful assessment tools including a case study
group response format for analyzing and responding to ethical-issue
case studies; a template for portfolio artifact summary and reflection
development; and templates for diagnosing and leveraging processes,
relationships, and group cultures within a complex system. Over a


three-year period, the course assessment data and course design
reflections have provided the impetus to transform program instruction
delivery from predominantly face-to-face to a hybrid or blended
format, with 49% of course instruction occurring on line.The updating
and improvement of course resources, objectives, and learning
activities to strengthen relevance of course learning is ongoing.
Intensive work has also been completed that pulls strands of learning
for social justice and ethical decision making across the courses.
Program faculty has also worked to ensure greater skill development
in the latest technologies related to course curriculum.
In addition to being part of the 360-degree feedback loop, the
Principal CAPS advisory group provides political and structural
support as it carries some of the ―freight‖ involved in change efforts –
carefully examining all perception and performance data, determining
the issues to address each year, uncovering strategies to meet
identified needs, and hosting a collective voice. To date, they have
recommended more of a practice to theory approach to program
design, a longer and more comprehensive internship experience, a
more customized recruitment and preparation approach for individual
school districts, a more in-depth selection process for program
participants, and maintenance of a blended rather than totally online
program delivery.Plans are underway to implement all of the
recommendations.
Improving the program has also included an ongoing search for
additional feedback sources and measurement tools.One example is
the need for measurement and feedback data to gauge outcomes
related to leadership effects on prek-12 school conditions and school
improvement. Housed at the National Center for the Evaluation of
Educational Leadership Preparation and Practice and based on the
NCATE ELCC standards, the School Leadership Preparation and
Practice (SLPPS) survey series has been chosen as a step towards
meeting this need. Developed to measure key concepts in a recentlydeveloped conceptual framework of leadership preparation inquiry, the
series includes the following: (1) a survey of preparation program


features to be completed by program faculty, (2) a school leader
survey for program completers and school leader alumni, and (3) a
survey for teachers who work with school leaders who are program
alumni (Pounder, 2012). It will be used for identification of areas for
program and course improvement; for development of a case for
program resources and support; and for research focused on the
relationship between program design/delivery, graduate outcomes, and
on the job school improvement work.

Conclusion
In summary, meaningful course and program assessment for a
school leadership preparation program is crucial to its continuous
improvement, particularly in view of the constantly shifting
requirements of certifying entities and the ever changing knowledge
and skill set associated with school leadership roles. For program
completers to make an impact as school leaders, they must engage in
learning that is relevant to prek-12 school environments, delivered in a
coherent and engaging sequence, and assessed by both internal and
external measures of effectiveness. Development, implementation,
and improvement of such a program has been the goal of program
faculty at a regional university. Informing the field of the faculty’s
journey extends the dialogue as it attempts to provide a realistic ―look‖
through a more comprehensive 360-degree feedback-driven case study
method that provides mindful support for continuous program
improvement.


References
Department of Educational Leadership and Policy Studies at Texas
A&M University-Central Texas.(n.d.).About us: School
leadership program mission statement. Retrieved from
http://www.ct.tamus.edu/departments/educationleadership/
index.php
Bracken, D., Dalton, M., Jako, R., McCauley, C., &Pollman, G.
(1997).Should 360-degree feedback be used on for
developmental purposes? Greensboro, NC: Center for Creative
Leadership.
Darling‐Hammond, L., LaPointe, M., Meyerson, D., & Orr, M.
(2009).Preparing school leaders for achanging world.San
Francisco, CA: Jossey-Bass.
Frye, B., Bottoms, G., & O’Neill, K. (2005).The principal internship:
How can we get it right? Atlanta, GA: Southern Regional
Education Board.
Fullan, M. (2009).Motion leadership: The skinny on becoming change
savvy. Thousand Oaks, CA: Corwin.
Glasman, N., Cibulka, & Ashby, D. (2002).Program self-evaluation
for continuous improvement.Educational Administration
Quarterly, 38, 257-288.
Glickman, C. (2003). Holding sacred ground. New York, NY: Wiley
and Sons.
Hargreaves, A., & Shirley, D. (2009). The fourth way: The inspiring
future for educational change. Thousand Oaks, CA: Corwin.
Hernez-Broome, G., & Hughes, R. (n.d.). Leadership development:
Past, present, and future [Electronic version]. Human Resource
Planning, 24-32. Retrieved from
http://www.ccl.org/leadership/pdf/research/cclLeadershipDevel
opment.pdf
Lashway, L. (2003). Transforming principal preparation.Eric Digest.
Eugene, OR: ERICClearinghouse on Educational
Management.
Levine, A. (2005). Educating school leaders.Princeton, NJ: Education
Schools Project.


Orr, M. T. (2009). Program evaluation in leadership preparation and
related fields.In M. D. Young & G. Crow (Eds.), Handbook of
research on the education of school leaders. New York, NY:
Taylor & Francis.
Orr, M. T., King, C., &LaPointe, M. ( 2010). Districts developing
leaders: Lessons on Consumer actions and program
approaches from eight urban districts. Boston, MA:
Education Development Center.
Pounder, D.(2012). School leadership preparation and practice survey
instruments and their uses.Journal of Research on Leadership
Education, 7(2), 254-274.
Robinson, V. (2010). From instructional leadership to leadership
capabilities: Empirical findings and methodological challenges.
Leadership and Policy in Schools, 9, 1-26.
Seashore Louis, K., Leithwood, K., Wahlstrom, K., &Anderson, S.
(2010). Investigating the links to improved student learning:
Final report of research findings.Minneapolis, MN:University
of Minnesota.
Sergiovanni, T. (2006). Rethinking leadership. Thousand Oaks, CA:
Sage.
Texas Education Agency.(2010). Texas examination of educator
standards TExES preparation manual 068.Retrieved from
http:texes.ets.org/assets/pdf/testprep_manuals/068_principal_8
2762_web.pdf
Texas Principals Leadership Initiative. (2002). The Texas leadership
preparation network lighthouse initiative: A continuous
improvement process for educational leadership programs.
Austin, TX: Author.
Tucker, P., Young, M., &Koschoreck, J. (2012). Leading researchbased change in educational leadership preparation: An
introduction. Journal of Research on Leadership Education,
7(2), 155-171.

Dr. Bobbie Eddins, Dr. Jeffrey Kirk, Dr. Dorleen Hooten, Dr. Brenda Russell - Published in NFEAS JOURNAL, 31(1) 2013-2014 - Dr. William Allan Kritsonis, Editor-in-Chief, NATIONAL FORUM JOURNALS (Founded 1982) - www.nationalforum.com

More Related Content

What's hot

Viewers also liked

Similar to Dr. Bobbie Eddins, Dr. Jeffrey Kirk, Dr. Dorleen Hooten, Dr. Brenda Russell - Published in NFEAS JOURNAL, 31(1) 2013-2014 - Dr. William Allan Kritsonis, Editor-in-Chief, NATIONAL FORUM JOURNALS (Founded 1982) - www.nationalforum.com

Recently uploaded

Dr. Bobbie Eddins, Dr. Jeffrey Kirk, Dr. Dorleen Hooten, Dr. Brenda Russell - Published in NFEAS JOURNAL, 31(1) 2013-2014 - Dr. William Allan Kritsonis, Editor-in-Chief, NATIONAL FORUM JOURNALS (Founded 1982) - www.nationalforum.com