A study of staff development using the Kirkpatrick Model

1. Leadership
Academy:
Application of the
Kirkpatrick
Evaluation Model
KAM MARVEL
ST. THOMAS UNIVERSITY

2. “It is only through evaluation that
value exists”
- FRIEDRICH NIETZSCHE

3. Kirkpatrick’s Model
 Through application of Kirkpatrick's four levels of reaction, learning,
behavior, and results, an effective professional development experience
can be created.
 Implementation of this model will require observing participant
engagement (reaction), assessment (learning), increased job performance
(behavior), and increased outcomes of the organization (results) (Rajeev, et.
al., 2009).

4. The Goal
 A medium-sized community-based organization desires to grow the skills of
part-time afterschool activity coordinators in order to create a pipeline of
future agency leaders.
 Agency leadership has tasked program directors with creating an impactful
experience, define short-term and long-term outcomes, propose a budget and
design, deliver and evaluate the project.

5. Leadership Academy
After careful research, including organizational data, learner analysis, and consultation with the
curriculum specialist, site directors, and principals, a leadership academy is designed using a problem-
based learning structure.
Activity coordinators who have been employed at least two years and received a recommendation
from their site director are offered a four month “academy” including bi-weekly meetings with
various agency staff and facilitated by the curriculum specialist (retired principal).
The commitment is volunteer and mirrors a college course structure with live meetings, assignments,
portfolios, group work, and a culminating product presented to agency leadership and site directors.

6. Level 1: Reaction
Program directors attend and observe each live-learning session
over the course of the Leadership Academy and perform the
following actions:
• Observation notes are taken
• Informal questioning of participants
The curriculum specialist performs “thumb's up / thumb’s down” at
the conclusion of each class.
At the final meeting, a formal survey is conducted with participants
including suggestions for improvement.

7. Level 2: Learning
Formative assessment is chosen for the Leadership Academy.
Strategies include:
• Monthly checks of portfolio work
• Each session contains report-outs after small-group work
• The curriculum specialist assesses individuals and offers
verbal and email feedback
• The final presentation offers a view to organizational
leadership of the final product, including each participant’s
understanding of leadership and presentation skills

8. Level 3: Behavior
Each participant who successfully graduated
from the leadership academy is tasked with
additional responsibilities the successive
school year.
In addition to formal observations each
month related to instruction, site coordinators
track weekly progress on new tasks (lesson
planning, data entry, parent communication,
team-building with peers) and offer informal
feedback

9. Level 4: Results
Data from each level is analyzed for impact.
 Student outcomes:
 Experience surveys
 Program participation data
 School-day grades, attendance, and behavior
Staff outcomes:
 Principal surveys
 Observations
 Annual employee performance evaluations

10. Level 4: Results (cont.)
Longitudinal data related to employee retention,
growth, and performance is collected.
Metrics include:
The number of participants in Leadership Academy
employed after three, five, and seven years
The number of participants in Leadership Academy
promoted to site director
The number of participants in Leadership Academy with a
minimum score of 4 out of 5 on annual employee
performance evaluations

11. Level 4: Results (cont.)
Annual evaluations of after-school programs performed by an independent external
evaluator are conducted and include performance measurements related to professional
development, staffing, budgeting, and student outcomes. The studies offer the
following conclusions:
Staff retention of part-time after school staff over the past five years has increased
every year
34 of 51 participants of Leadership Academy have been promoted at least once
Student outcomes have shown statistically significant improvement from the
beginning of the project to the current year.

12. Conclusion
The success of the project dictates continuation of the
Leadership Academy.
Extension of the evaluation results in analyzing a return
on investment in Leadership Academy.
The organization determines that other than compliance
mandated training, Leadership Academy is the most
necessary professional development as the cost per
participant saves money in human resources and results
in stronger outcomes, further resulting in the
procurement of additional grants.

13. References
Rajeev, P., Madan, M. and Jayarajan, K. (2009). Revisiting Kirkpatrick’s model – an
evaluation of an academic training course. Current Science, 96(2).