2. Overview of the Ohio Resident Educator
Program
RE Program Evaluation Design and Plan
Resident Educator Program Year 1 and
Beyond
Changing the Landscape of Teaching in
Ohio
Discussion
3. Miami University, Evaluation & Assessment Center
– Dr. Sarah Woodruff
University of Cincinnati, Evaluation Services
Center – Dr. Debbie Zorn; and College of
Education, Criminal Justice and Human Services
– Dr. Julie Morrison
Ohio University, Voinovich School of Leadership
and Public Affairs – Dr. Marsha Lewis and Dr.
Holly Raffle
4. Substantial investment in development and
implementation
Resource and infrastructure needs will continue to
escalate as program grows to scale by Fall 2014
Significant component of Ohio’s RttT commitment to
provide great teachers and great leaders for all Ohio
schools
Designed to improve teacher retention, enhance teacher
quality and effectiveness, result in improved student
achievement
Will serve as many as 20,000 Resident Educators across
nearly 1,000 traditional and community school LEAs
6. Resident Educator License
2009: New licensure system
2011: First Resident Educator licenses issued
Ohio Resident Educator Program
2009-2010 and 2010-2011: Transition from Entry
Year Program to Resident Educator Program
2011-2012: Ohio Resident Educator Program
begins
7. Teaching and Learning Cycle
State Mentor Training
Formative Assessment
Monitor and Document REs’ Progress
Summative, Performance-based
Assessment
8. In Year 1, RE:
Communicates with mentor and principal
Uses formative assessment activities to collect
evidence, advance practice
Uses state-designed formative assessment tools
Documents instructional meetings with mentor
Maintains a collection of evidence/documents to inform
summative assessment
In Year 2, RE:
Continues processes, protocols, and tools from Year 1
Utilizes differentiated support and resources to move
toward meeting Year 2 goals
Completes Year 2 Formative Progress Review
9. In Year 3, RE:
Prepares for summative assessment
Completes summative assessment
In Year 4, RE:
Participates in PD and leadership activities
Re-takes deficient portions of summative assessment
Completes RE Program requirements and applies for
5-year professional license
10. The LEA:
Provides district/school orientation
Determines eligibility and registers REs
Selects, assigns, and supports mentors
Provides time and support for mentor training
Provides support and PD for REs and mentors
The Principal:
Works with PC to select/assign mentors
Ensures mentors attend state training
Provides time for mentor-RE collaboration
Provides opportunities for reciprocal observations
Collaborates with mentor and RE to align RE goals
11. The Mentor:
Attends all required training for certification
Communicates with RE and building administrator
Respects confidential relationship with RE and
principal
Supports RE through use of formative assessment
processes, protocols, and tools
Collaborates with the PC to complete end-of-year
Formative Progress Review
13. The four primary evaluation foci include:
Implementation fidelity and compliance
Implementation quality and effectiveness
Implementation influence and impact
Implementation scale-up and sustainability
14. Objective 1 - State’s Role in the RE Program
Objective 2 - LEA Implementation of the RE Program
Objective 3 - Impact of the RE Program on Resident
Educators
Objective 4 - Impact of the RE Program on Resident
Educator Mentors
Objective 5 - Impact of the RE Program on LEA
Administrators, Policies and Procedures
Objective 6 - Full Implementation of the RE Program
◦ Part 1: Implementation and Reporting of RE Summative
Assessment
◦ Part 2: Resource Analysis for Sustainability of Full Implementation
15. Evaluation Focus Overarching Evaluation Questions
Implementation To what extent did To what extent did To what extent did
Fidelity and the State develop LEAs implement a REs, RE mentors,
Compliance an infrastructure local REP aligned and principals
and provide with the State utilize guidance
support to facilitate REP? and materials
LEAs in Objective 2 provided by the
implementing local State REP and/or
REP with fidelity to the LEA REP?
the State REP? Objective 3, 4, 5
Objective 1, 6
16. Evaluation Focus Overarching Evaluation Questions
Implementation To what extent To what extent do To what extent do
Quality and does the State’s LEAs meet the REs, RE mentors,
Effectiveness implementation of State’s quality and principals
the REP meet its standards for local perceive that the
own and external REP LEA REP has
quality standards? implementation? been implemented
Objective 1, 6 Objective 1, 2 effectively and
with quality?
Objective 1, 2
17. Evaluation Focus Overarching Evaluation Questions
Implementation What is the nature What is the nature What is the nature
Influence and and extent of the and extent of and extent of
Impact State REP influence of the impact of the LEA
contribution to LEA REP on local REP on REs, RE
State RttT goals policy, procedures, mentors, and
regarding educator and practices principals? What
effectiveness? regarding support are the anticipated
What are the for REs? What are and unanticipated
anticipated and the anticipated outcomes of
unanticipated and unanticipated implementation?
outcomes of outcomes of Objective 3, 4, 5
implementation? implementation?
Objective 1, 6 Objective 2, 5
18. Evaluation Focus Overarching Evaluation Questions
Implementation What are the What are the What are the
Scale-up and benefits and trade- benefits and trade- benefits and trade-
Sustainability offs for the State offs for the LEA of offs for the REs,
of sustaining the sustaining the LEA RE mentors, and
State REP in its REP in its current principals of
current form? form? sustaining the LEA
Objective 6 Objective 2, 6 REP in its current
form?
Objective 3, 4, 5,
6
19. Concurrent nested, mixed-methods design
(Creswell & Plano Clark, 2007)
Case study methodology (Merriam, 1998)
Collective case study (Stake, 2000) of 30 LEAs
across Ohio
Typical case and stratified purposeful sampling
(Patton, 2002)
Triangulation model with data transformation
(Creswell & Plano Clark, 2007)
Collection/analysis of survey data
20. State Level LEA Level Educator Level
Key ODE RE Program Staff LEA RE Program Resident Educators
Informants RE Program State TrainersCoordinator RE Mentors
LEA Superintendent/ LEA Administrators
Designee
Data State REP documents NTC Surveys NTC Surveys
Sources State REP tools LEA Annual Reports Building REP documents
State REP communications LEA REP documents Building REP
Observations of training LEA REP communications
Interviews communications Focus Groups
Teacher induction Interviews Interviews
documents from other RE Questionnaire
states
21. # of REs in all # of REs in all # of RE # of
# of LEAs to Average # of
case study case study Mentors in all Administrator
Size of LEA be selected as REs per year
LEAs LEAs case study s in all case
cases in each LEA
2012-13 2013-14 LEAs study LEAs
Small – rural 10 to 12 3 60 to 72 90 to 108 10 to 24 10 to 36
and small
town
Medium – 5 to 6 8 80 to 96 120 to 144 10 to 42 10 to 42
small urban
and suburban
Large – large 5 to 6 15 150 to 180 225 to 270 15 to 36 10 to 60
urban and
suburban
Joint 1 to 2 5 10 to 20 15 to 30 1 to 2 1 to 2
Vocational
LEA
Community 4 to 7 3 24 to 42 36 to 63 4 to 21 4 to 21
Schools LEA
Total 25 to 33 162 to 205 324 to 410 486 to 615 40 to 125 35 to 161
23. Total Number of REs = 4,206
Number of Active RE Mentors = 2,975
Number of LEAs implementing REP = 860
24. Resident
Educators
Other eCommunity
7% 3% throughout
Private
Ohio
12%
Community
Traditional
17%
61%
Traditional 2,558
Community 729
Private 500
Other 311
eCommunity 108
Total 4,206
25. Local Education Agencies with Resident Educators
Traditional District LEAs 454
Community School LEAs 227
Private/Parochial LEAs 84
Other LEAs (≈ESCs, CTCs) 95
Identified Resident Educator Mentors
Type # Mentors REs/Mentor
Traditional District LEAs 1,979 1.29
Community School LEAs 363 2.26
Private/Parochial LEAs 363 1.34
Other LEAs (≈ESCs,
270 1.15
CTCs)
Total 2,975 1.40
26. Examples of Teacher Turnover Rates
◦ Traditional District LEAs
Large Urban/Low SES 11 RE / 3,187 FTE ≈ 0.03%
Rural/Moderate SES 13 RE / 457 FTE ≈ 2.8%
Rural/Low SES 3 RE / 20 FTE ≈ 15.0%
◦ Community School LEAs
Large Urban Magnet School 3 RE / 42 FTE ≈ 7.1%
Large Urban High School 11 RE / 40 FTE ≈ 27.5%
Large Urban Credit Recovery 8 RE / 11 FTE ≈ 72.7%
27. By Fall 2012, two cohorts of REs (approximately
8,000) will be teaching in Ohio schools
In 2011-2012 more than 4,000 RE mentors were
trained and nearly 3,000 mentored REs. On
average, each mentor worked with 1 or 2 REs.
More mentor trainings will take place in Summer
2012.
REs are widely and unequally distributed. More
than half of Ohio’s districts are small rural districts.
Over 25,000 teachers teach in moderate to large
urban schools, while nearly 30,000 teach in
suburban schools.
33. How does the RE Program intersect with:
Student Growth Measures (Value-Added)
Ohio Teacher Evaluation System
Ohio Principal Evaluation System
Teacher Incentive Program
TeachOhio
Other?
Focus on Teacher Effectiveness as the Key to
Improve Student Achievement
34. How should Ohio operationalize teacher
effectiveness? What might be appropriate
indicators of teacher effectiveness?
How might implementation of the Resident
Educator Program in districts change the culture
of teacher professional development and learning
for all teachers?
What organizational, structural, or policy changes
might be necessary to support and sustain such a
change in culture?
35. Dr. Sarah Woodruff Dr. Marsha Lewis
woodrusb@muohio.edu lewism5@ohio.edu
513.529.1686 740.593.1435
Dr. Debbie Zorn Dr. Holly Raffle
zorndl@ucmail.uc.edu raffle@ohio.edu
513.556.3818 740.597.1710
