OERC RESEARCH RELATED
TO STUDENT GROWTH
MEASURES AND EDUCATOR
EFFECTIVENESS
Jill Lindsey, Ph.D.
Wright State University
Ma...
 OERC can examine statewide policy and
practice questions
 Follow and document early implementation
in order to inform p...
 Time span of findings offer insight into changing landscape
around teacher evaluation and student growth measures
 2012...
4
METHODOLOGY
 Structured interviews with superintendents
and administration team members
 Focus groups with teachers
 ...
 OTES/OPES Implementation Study (37 LEAs)
 Extended Testing for Value-Added Reporting (23
LEAs)
 Initial Use of Student...
 Sequencing, planning, feedback, and student
growth measures for teachers and principals
 Preparation for evaluation
 E...
7
EARLIEST FINDINGS
 Generally positive about the new evaluation systems
 Supported use of student growth measures in ev...
 Grant funds provided vendor testing for grades 1,2,3, and
high school subjects
 Provided teacher-level value-added scor...
Findings
 Want reliable student
growth measures
 Lack assessment literacy
 Unclear how vendors will
provide data
 Unce...
10
INITIAL USE OF STUDENT
LEARNING OBJECTIVES
Study examined fidelity of SLO use for:
 improving student performance
 me...
 Training was not uniform across the state
 Assessments varied widely across grade
levels, buildings, and districts
 Pr...
12
SGM POLICY AND PRACTICE
STUDY
OERC study of early adopter districts of
Student Growth Measures
 Designed to provide ti...
Focus group themes:
 Fairness questions (e.g. Category A teachers do
not know OAA items in advance while Category C
teach...
14
SGM POLICY AND PRACTICE
SURVEY
 Deployed late February through mid-April, 2014
 22% response rate (603 teacher respon...
 Early stage of implementation
 Uncontrollable factors
 Unequal measures/accuracy of the measures
 SLOs teacher-develo...
Teachers who see value in SGMs:
 Feel it is important to measure student growth
 Recognize the need for accountability
...
“I do think it is important to make sure a child makes
adequate growth. However, there are factors that are
out of my cont...
Teacher–Student Data Link/Roster Verification is necessary to
ensure SGM data quality.
Research Questions:
 Are teachers ...
19
TEACHER ROSTER
VERIFICATION SURVEY
Sent online survey to all teachers and principals who
completed the link/roster veri...
2011 2013 2014*
(prelim.)
Yes 46% 57% 59%
No 23% 25% 25%
Don't
know
31% 17% 16%
Teachers – Do you think the linkage proces...
21
TEACHER ROSTER
VERIFICATION SURVEY
For teachers that answered “No”:
Teachers – Explain why you think the student–teache...
2011 2013 2014*
(prelim.)
Not at all
confident
39% 32% 35%
Somewhat
confident
55% 61% 58%
Very
confident
6% 8% 7%
Teachers...
Concerns
 Early in
implementation—lack of
trust and
misunderstandings
 Perceived unfairness of
different kinds of
measur...
 Build trust by continuing to include teachers and
administrators in conversations and policies that impact
them
 Acknow...
QUESTIONS?
JILL.LINDSEY@WRIGHT.EDU
LEWISM5@OHIO.EDU
connect@oerc.osu.edu | oerc.osu.edu
PLANNING FOR THE FUTURE,
LEARNING FROM THE PAST: WHAT
CAN SCHOOLS LEARN FROM COLLEGE
AND CAREER PROFILES OF
GRADUATES?
Jos...
Ohio’s Constitution
A “thorough” and
“efficient“
education.
OHIO’S STANDARDS OF PUBLIC
EDUCATION
College for all
High
School
College Work
Education and Career
High
School
• CTE
• STEM
College
• AP/Dual
Enrollment
• Scho...
30
CHANGING VIEWS OF WORK
REQUIRE NEW INFORMATION
 As educators, we want to know about a variety of
educational outcomes,...
31
QUALITY OF OUTCOMES MATTER
 In this day and age, the quality of the outcomes matter a
great deal, and, therefore, we a...
32
PILOT HIGH SCHOOL REPORTS
 Using data from the Ohio Longitudinal Data Archive
(OLDA), the OERC has been able to answer...
33
PILOT REPORTS: COLLEGE AND
CAREER
34
WHAT KIND OF EMPLOYMENT
EXPERIENCES DO STUDENTS HAVE
AFTER SCHOOL?
35
WHAT HIGHER EDUCATION
OUTCOMES DO STUDENTS
HAVE?
36
HOW WELL DO SCHOOLS
COMPARE WITH EACH OTHER IN
REMEDIAL EDUCATION?
37
VISUAL LOOK
College and Career Report
High School Name
County
State
 
How do X year graduates from this school compare ...
38
COLLEGE OUTCOMES
How do X year graduates from this school compare to others in Ohio?
Education
(Note): Following legal ...
39
COLLEGE OUTCOMES (CONT.)
Percent of graduates ready for college
Average state scholarship awards for graduates
Overall ...
40
CAREER OUTCOMES
Number of Students working in Ohio in the year following graduation
Average annual earnings for individ...
 Develop a formal high school college and career report for
select districts (next up, Columbus City Schools; Battelle
Fo...
THANKS FOR YOUR
ATTENTION!
connect@oerc.osu.edu | oerc.osu.edu
SM22
Slide 42
SM22 to be consistent with brochure and briefs
www.oerc.osu.edu | connect@oerc.osu.edu
Sunny Munn, 6/13/2013
THIRD GRADE READING
GUARANTEE:
A CASE STUDY
Suzanne Franco, Professor, Wright State University
Jarrod Brumbaugh, Principal...
 Ohio Third Grade Reading Guarantee (TGRG)
2012
 In 2012–13, 81% of Ohio’s 3rd graders were
proficient or above
 ODE of...
 Co-located in Midwestern Ohio but had
not collaborated on previous initiatives
 Orton-Gillingham Multi-Sensory training...
47
CONSORTIUM DEMOGRAPHICS
LEA Typology Report
Card
Rating
2010–
2011
%
passed
3rd grade
reading
2011–
2012
% passed
3rd g...
 Feedback and buy-in for the training,
implementation, and PLC.
 Progress and monitoring tools used.
 Reading skills im...
For each LEA:
 Document analysis of historical data and end-of-year
2014 RIMPS
 Interviews and focus groups with Adminis...
51
O/G TRAINING
Training Details
 Two 5-day sessions the week after school year ended
 One 5-day session in November
 R...
52
IMPLEMENTATION
Implementation Details
 LEA 1 – O/G not required due to receipt of grant
funds in mid-September, 2013. ...
Implementation Feedback Details
 Not all supplies at beginning of year for all teachers due to
delay in receiving grant f...
54
PROGRESS AND MONITORING
TOOLS
 LEA 1: DIBELS
 LEA 2: NWEA (2012); DIBELS (2013)
Feedback
 O/G assessment tools not O...
Successes
 After School
Program
 Students respond
well to Multi-Sensory
 Teachers want more
training
Challenges
 Use o...
 LEA1
 Grade 3 results to date
 Changes in implementations for 2014–2015
 LEA 2 Details
 Grade 3 results to date
 Ch...
 Assessment tools aligned with TGRG programs
funded by state need approval.
 Primary students exhibit high anxiety regar...
 Continue funding for TGRG development.
 Continue monitoring LEA implementation of
funded and non-funded TGRG
implementa...
QUESTIONS
suzanne.franco@wright.edu
BrumbaughJ@milton-union.k12.oh.us
thompsond@piqua.org
Making Research Work for Educati...
READY OR NOT?
E x t e n d i n g Y o u r K n o wl e d g e T h r o u g h R e s e a r c h T h a t W o r k s ! I C o l u m b u...
 Research and study
 Use of multiple forms of data
 Create a culture of “calculated risk-takers”
 Embed professional d...
 Role model to all at all times
 Establish high expectations
 Collaborate, share
 Trust, be truthful and supportive
 ...
64
EXPLORE OPPORTUNITIES
 Through grants, pilot studies, action
research
 Connect with:
 Personnel at colleges and univ...
 Milton-Union was involved in a number of
grants:
 RttT Mini-grant Value-Added
 Student Growth Measures
 Early Literac...
Our culture and the research and data from
our grant involvement led to development
of our OTES instrument, and a successf...
All initiatives impact one another and YOU:
 OTES
 OPES
 Graduation requirements
 Third Grade Reading Guarantee
Keep t...
 Do the research upfront
 Study the data
 Reflect, revise if necessary
 Building project
 Food service program
68
CAL...
 All day, every day kindergarten
 On-site Head-Start programs
 Grouping students by quintiles
 H.S. ACT EOC exams
 Re...
 How do we better prepare students for their
futures – colleges, universities, employers?
 How can we convey to young pa...
QUESTIONS?
RAMMELV@MILTON-UNION.K12.OH.US
connect@oerc.osu.edu | oerc.osu.edu
Oerc june 2014 final ppt combined
Oerc june 2014 final ppt combined
Oerc june 2014 final ppt combined
Upcoming SlideShare
Loading in …5
×

Oerc june 2014 final ppt combined

338 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
338
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Oerc june 2014 final ppt combined

  1. 1. OERC RESEARCH RELATED TO STUDENT GROWTH MEASURES AND EDUCATOR EFFECTIVENESS Jill Lindsey, Ph.D. Wright State University Marsha Lewis, Ph.D. Ohio University E x t e n d i n g Y o u r K n o wl e d g e T h r o u g h R e s e a r c h T h a t W o r k s ! I C o l u m b u s , O H I J u n e 1 8 , 2 0 1 4
  2. 2.  OERC can examine statewide policy and practice questions  Follow and document early implementation in order to inform policy and practice  Research implementation for multiple years as start-up issues are resolved and implementation takes hold—look for what is working and what can be improved. PURPOSE OF RESEARCH
  3. 3.  Time span of findings offer insight into changing landscape around teacher evaluation and student growth measures  2012: Teachers and principals philosophically supportive of new evaluation system and need to measure student growth  2013: HB 555  2014: Teachers and principals far less supportive and troubled by concerns related to use of different types of SGMs for evaluation  Common Themes 3 FOUR STUDIES SPANNING THREE YEARS
  4. 4. 4 METHODOLOGY  Structured interviews with superintendents and administration team members  Focus groups with teachers  Surveys of teachers in each pilot LEA  eTPES data analysis
  5. 5.  OTES/OPES Implementation Study (37 LEAs)  Extended Testing for Value-Added Reporting (23 LEAs)  Initial Use of Student Learning Objectives (30 LEAs)  Student Growth Measures Policy & Practice (13 LEAs) 5 FUNDED PROJECTS RELATED TO STUDENT GROWTH MEASURES
  6. 6.  Sequencing, planning, feedback, and student growth measures for teachers and principals  Preparation for evaluation  Experiences of teachers and principals evaluated using student growth measures  Processes and measures of student growth that districts adopted for use in teacher and principal evaluation systems 6 OTES/OPES IMPLEMENTATION STUDY
  7. 7. 7 EARLIEST FINDINGS  Generally positive about the new evaluation systems  Supported use of student growth measures in evaluation  Lack of trust & misunderstandings about value-added, vendor assessments, and local measures of student growth  Unfairness of using different kinds of measures and differing time cycles for different measures of student growth  Conversations around new evaluation system focus on instruction  Time required to complete evaluation took time away from working with students  Appreciative at being asked about their experiences and views
  8. 8.  Grant funds provided vendor testing for grades 1,2,3, and high school subjects  Provided teacher-level value-added scores from vendor test results  Processes and challenges related to extended testing implementation  Role of roster verification  Use of SGMs in educator evaluation  Best practices/lessons learned 8 SGM EXTENDED TESTING MINI- GRANT
  9. 9. Findings  Want reliable student growth measures  Lack assessment literacy  Unclear how vendors will provide data  Uncertain of roster verification timing and impact on VAM  LEAs opting to use lowest percentages in weights  Grateful for being asked about their experiences Nine Drop-outs’ Reasons  Requirement to use extended testing results too soon, unfair, not part of grant  Cost of extended testing was too high  Too many changes, and too much on teachers’ plates SGM MINI–GRANTS FOR EXTENDED TESTING
  10. 10. 10 INITIAL USE OF STUDENT LEARNING OBJECTIVES Study examined fidelity of SLO use for:  improving student performance  measuring academic growth  evaluating teachers
  11. 11.  Training was not uniform across the state  Assessments varied widely across grade levels, buildings, and districts  Processes excessively time-consuming  More challenging for semester or quarter courses; limited time to complete the pre- test, teach, post-test cycle  Implementation hampered by too many changes to common core, piloting new state tests and PARCC, and implementing OTES  Many emotional moments and gratitude for being invited to talk about experiences Interviews Surveys Documents eTPES Data 11 EARLY SLO THEMES ALL DATA NOT YET ANALYZED
  12. 12. 12 SGM POLICY AND PRACTICE STUDY OERC study of early adopter districts of Student Growth Measures  Designed to provide timely data to inform state policy and district practice.  “What does this look like when implemented?”  Teachers’ perceptions of SGM components  Do SGMs correlate with Performance on Standards? If not, why not?  The distribution of teacher and principal ratings
  13. 13. Focus group themes:  Fairness questions (e.g. Category A teachers do not know OAA items in advance while Category C teachers develop their own assessments)  Principals’ time consumed with teacher observation activities  Teachers have questions/misconceptions about value-added methodology 13 SGM POLICY AND PRACTICE STUDY
  14. 14. 14 SGM POLICY AND PRACTICE SURVEY  Deployed late February through mid-April, 2014  22% response rate (603 teacher respondents/2,709 full- time teachers) N = 469–classroom teachers, 97 intervention specialists  Survey responses were similar to focus group findings  Of the four SGMs (Value-Added, SLOs, Vendor-Approved Assessments, Shared Attribution), more surveyed teachers think Student Learning Objectives “most accurately assess a teacher’s instructional impact.”
  15. 15.  Early stage of implementation  Uncontrollable factors  Unequal measures/accuracy of the measures  SLOs teacher-developed, validity/reliability questions  Others see the SLOs as most fair because focused on the content taught and results during evaluation year  Approved vendor assessments may not match content standards  Value-added model was not formulated to measure individual teacher effectiveness 15 FAIRNESS CONCERNS WITH SGMS & EVALUATION
  16. 16. Teachers who see value in SGMs:  Feel it is important to measure student growth  Recognize the need for accountability  SGMs useful source of feedback for planning and adjustment to outcomes 16 SGM POLICY AND PRACTICE STUDY
  17. 17. “I do think it is important to make sure a child makes adequate growth. However, there are factors that are out of my control (attendance, home support, etc.) that affect a child's learning and are not considered when calculating the yearly academic growth of a student.” “It shows the effectiveness of a teacher and useful data to adjust your teaching.” 17 SGM POLICY AND PRACTICE STUDY
  18. 18. Teacher–Student Data Link/Roster Verification is necessary to ensure SGM data quality. Research Questions:  Are teachers actively participating in the verification of their own rosters and percentage of instructional time with students as specified by Ohio’s roster verification process guidelines?  Do principals and teachers have access to adequate training and technical assistance?  Do principals and/or teachers perceive any issues with roster verification?  What do Ohio educators view as the hallmarks of a good system? 18 TEACHER ROSTER VERIFICATION RESEARCH
  19. 19. 19 TEACHER ROSTER VERIFICATION SURVEY Sent online survey to all teachers and principals who completed the link/roster verification process in spring 2013 and spring 2014 2013 survey: 5,984 teacher responses from 695 LEAs 2014 survey to-date: 6,778 teacher responses. Survey still in field.
  20. 20. 2011 2013 2014* (prelim.) Yes 46% 57% 59% No 23% 25% 25% Don't know 31% 17% 16% Teachers – Do you think the linkage process accurately captured what was happening in your classroom (i.e. students you taught last year, their length of enrollment, and your percentage of instructional time with them)? 20 TEACHER ROSTER VERIFICATION SUR
  21. 21. 21 TEACHER ROSTER VERIFICATION SURVEY For teachers that answered “No”: Teachers – Explain why you think the student–teacher linkage process did not accurately capture what was happening in your classroom. (open-ended) Themes:  Difficulty dividing time in various co-teaching situations.  Unable to account for student absences  Teachers want to be able to report finer increments of shared instructional responsibility  Students’ schedules changed too often/environment too dynamic to accurately estimate time
  22. 22. 2011 2013 2014* (prelim.) Not at all confident 39% 32% 35% Somewhat confident 55% 61% 58% Very confident 6% 8% 7% Teachers – Given your experience with the linkage process, how confident are you that the linkage process improves the accuracy of the teacher-level value-added data? 22 TEACHER ROSTER VERIFICATION SURVEY
  23. 23. Concerns  Early in implementation—lack of trust and misunderstandings  Perceived unfairness of different kinds of measures  Time required to complete evaluation  Too many changes at the same time Kudos  Support measuring student growth  Training for assessment literacy desired  Appreciate being consulted and heard  Roster verification process is improving 23 COMMON THEMES ACROSS TIME
  24. 24.  Build trust by continuing to include teachers and administrators in conversations and policies that impact them  Acknowledge concerns as legitimate  Provide professional development opportunities to correct misunderstandings and knowledge deficits  Streamline paperwork where possible; use adaptations from the field  Modify policy and roll-out timelines when possible 24 RECOMMENDATIONS
  25. 25. QUESTIONS? JILL.LINDSEY@WRIGHT.EDU LEWISM5@OHIO.EDU connect@oerc.osu.edu | oerc.osu.edu
  26. 26. PLANNING FOR THE FUTURE, LEARNING FROM THE PAST: WHAT CAN SCHOOLS LEARN FROM COLLEGE AND CAREER PROFILES OF GRADUATES? Joshua D. Hawley Director, OERC and Associate Professor John Glenn School of Public Affairs The Ohio State University Making Research Work for Education
  27. 27. Ohio’s Constitution A “thorough” and “efficient“ education. OHIO’S STANDARDS OF PUBLIC EDUCATION
  28. 28. College for all High School College Work Education and Career High School • CTE • STEM College • AP/Dual Enrollment • School + Work Workforce Training • Apprenticeship • Military 29 LINKING SCHOOL TO WORK
  29. 29. 30 CHANGING VIEWS OF WORK REQUIRE NEW INFORMATION  As educators, we want to know about a variety of educational outcomes, not just college-going rates for students. At its broadest, we might consider the following domains:  College (traditional two and four-year sectors)  Credentialed and non credentialed workforce training  Apprenticeships  Military
  30. 30. 31 QUALITY OF OUTCOMES MATTER  In this day and age, the quality of the outcomes matter a great deal, and, therefore, we are concerned with how well students are prepared to perform over time.  Concerns we typically have in this case:  Student remediation  Does student knowledge match what is required in college classes?  Are students prepared to pick a career? (I distinguish this from a job)  What happens to kids that go directly from high school to work?  What happens to kids that dropout (both in terms of further education and work?)
  31. 31. 32 PILOT HIGH SCHOOL REPORTS  Using data from the Ohio Longitudinal Data Archive (OLDA), the OERC has been able to answer many of these questions, beginning for high schools, and present them in a format that schools can use.  The report has four key question areas:  What are the employment outcomes of high school graduates?  What are the post secondary education outcomes of high school graduates?  What is the quality of post secondary education high school graduates are carrying out?  What happens to individuals that do not graduate from high school (dropout)?
  32. 32. 33 PILOT REPORTS: COLLEGE AND CAREER
  33. 33. 34 WHAT KIND OF EMPLOYMENT EXPERIENCES DO STUDENTS HAVE AFTER SCHOOL?
  34. 34. 35 WHAT HIGHER EDUCATION OUTCOMES DO STUDENTS HAVE?
  35. 35. 36 HOW WELL DO SCHOOLS COMPARE WITH EACH OTHER IN REMEDIAL EDUCATION?
  36. 36. 37 VISUAL LOOK College and Career Report High School Name County State   How do X year graduates from this school compare to others in Ohio? Education Career Outcomes (Note): Following legal agreement covering data use by the OERC, individual cells with fewer then 10 people have been redacted. They are indicated by a *. School District State   Number of Students that Started High School in X year Number of Students working in Ohio in the year following graduation Number of Graduates Average annual earnings for individuals with high school diploma Number of Dropouts Average annual earnings for individuals without a high school diploma Average High School GPA       Percent of students in this class eligible for free or reduced lunch Industry of employment for graduates of high school not in college   Retail 20 Average junior year ACT scores for this class by subject   Constructi 20 Financial S 60 English   Math   Reading Science Composite Percent of graduates ready for college Industry of employment for high school dropouts   Retail 40 Average state scholarship awards for graduates Constructi 40 Financial S 10 Overall post secondary rate for graduates from this class Other 10 Two year college Four year college Other vocational/workforce training Percent of graduates from this class who attended an in‐state college or university Percent of graduates from this class who attended an out of state college or university Trend in college going rates for this school vs. state and district Working and school *Readiness scores are calculated based on the individual ACT score over or below the "Remeditaion Free Standard" Footer: Describe data origin. Refer to website that summarizes origin.  Retail Constr uction Retail Construc tion Financial Services Other
  37. 37. 38 COLLEGE OUTCOMES How do X year graduates from this school compare to others in Ohio? Education (Note): Following legal agreement covering data use by the OERC, individual cells with fewer then 10  people have been redacted. They are indicated by a *. School District State Number of Students that Started High School in X year Number of Graduates Number of Dropouts Average High School GPA Percent of students in this class eligible for free or reduced lunch Average junior year ACT scores for this class by subject English Math Reading Science Composite
  38. 38. 39 COLLEGE OUTCOMES (CONT.) Percent of graduates ready for college Average state scholarship awards for graduates Overall post secondary rate for graduates from this class Two year college Four year college Other vocational/workforce training Percent of graduates from this class who attended an in‐state college or university Percent of graduates from this class who attended an out of state college or university Trend in college going rates for this school vs. state and district
  39. 39. 40 CAREER OUTCOMES Number of Students working in Ohio in the year following graduation Average annual earnings for individuals with high school diploma Average annual earnings for individuals without a high school diploma       Industry of employment for graduates of high school not in college   Retail 20   Constructi 20 Financial S 60     Industry of employment for high school dropouts   Retail 40 Constructi 40 Financial S 10 Other 10 Retail Constr uction Retail Construc tion Financial Services Other
  40. 40.  Develop a formal high school college and career report for select districts (next up, Columbus City Schools; Battelle For Kids)  Complete Workforce Success Measures Project with the Office of Workforce Transformation (see OWT website for introduction: http://workforce.ohio.gov/ ).  Work with Ohio Department of Education and Board of Regents to answer questions about employment outcomes for K-12 and higher education. 41 FUTURE PLANS
  41. 41. THANKS FOR YOUR ATTENTION! connect@oerc.osu.edu | oerc.osu.edu SM22
  42. 42. Slide 42 SM22 to be consistent with brochure and briefs www.oerc.osu.edu | connect@oerc.osu.edu Sunny Munn, 6/13/2013
  43. 43. THIRD GRADE READING GUARANTEE: A CASE STUDY Suzanne Franco, Professor, Wright State University Jarrod Brumbaugh, Principal, Milton-Union Schools Making Research Work for Education Extending Your Knowledge Through Resear ch That Works! I Columbus , OH I June 18, 2 0 14
  44. 44.  Ohio Third Grade Reading Guarantee (TGRG) 2012  In 2012–13, 81% of Ohio’s 3rd graders were proficient or above  ODE offered competitive funding grant for developing TGRG 2013–2014 implementation  OERC funded a case study of a funded TGRG three-LEA consortium for 2013–2014 BACKGROUND 45
  45. 45.  Co-located in Midwestern Ohio but had not collaborated on previous initiatives  Orton-Gillingham Multi-Sensory training and instructional strategies  Professional Learning Community (PLC)  Parent Informational Opportunities 46 CONSORTIUM TGRG PLAN
  46. 46. 47 CONSORTIUM DEMOGRAPHICS LEA Typology Report Card Rating 2010– 2011 % passed 3rd grade reading 2011– 2012 % passed 3rd grade reading 2012– 2013 Mobilit y % White/ Non- Hispani c % Econ. Dis- advan. 1 2: Rural w/ avg. pvrty & small ADM Exc. 87.3 85.9 7.0 97 40 2 4: Small Town w high pvrty & avg ADM Exc. 90.3 80.2 11.4 87 53 3 2: Rural w/ avg. pvrty & small ADM Exc. w/ Distinc- tion 84.7 84 6.1 98 19
  47. 47.  Feedback and buy-in for the training, implementation, and PLC.  Progress and monitoring tools used.  Reading skills improved for On-Target students. For Not on Target students?  Percentage of K–3 students Not on Target in 2012-14? 49 RESEARCH QUESTIONS
  48. 48. For each LEA:  Document analysis of historical data and end-of-year 2014 RIMPS  Interviews and focus groups with Administrators (6), Teachers (12)  Observations of O/G training and classroom instruction 50 METHODOLOGY
  49. 49. 51 O/G TRAINING Training Details  Two 5-day sessions the week after school year ended  One 5-day session in November  Refresher course available summer, 2014 Training Feedback  Teachers felt it was engaging but too long, or covered grade levels not in their interest. They would like to repeat after one year of implementation.  Administrators from one LEA attended training. They felt that common language helped with classroom observations.
  50. 50. 52 IMPLEMENTATION Implementation Details  LEA 1 – O/G not required due to receipt of grant funds in mid-September, 2013. Used in RTI, Title 1, and other interventions.  KRAL is the identifier for K; State assessment tool for grades 1–3  DIBELS is the progress monitor along with STAR and Study Island  LEA 2 – O/G not required (see above). Used in RTI, intervention, and Title 1  NWEA (2012) and DIBELS (2013) for K–3  DIBELS is the progress monitor  LEA 3 decided not to participate
  51. 51. Implementation Feedback Details  Not all supplies at beginning of year for all teachers due to delay in receiving grant funds  Not all training completed at beginning of year (new teachers)  Use of O/G not required; inconsistency a challenge for teams  Merging O/G with LEA-approved reading curriculum difficult  Parent Nights were not well attended; PLC not formed 53 IMPLEMENTATION FEEDBACK
  52. 52. 54 PROGRESS AND MONITORING TOOLS  LEA 1: DIBELS  LEA 2: NWEA (2012); DIBELS (2013) Feedback  O/G assessment tools not Ohio-approved, therefore the LEAs use DIBELS and NWEA to assess student progress  RIMPs not standardized among LEAs (issue for moving and determining LEA or statewide impact)  For highly mobile student populations, 30-day requirement for RIMP is very difficult to meet.  Too much testing for young students; test anxiety rising
  53. 53. Successes  After School Program  Students respond well to Multi-Sensory  Teachers want more training Challenges  Use of O/G not consistent  Costs to sustain  O/G assessments not state approved  RIMP forms could be improved; data should be collected for analyses  No information about other LEA TGRG plans 55 CONSORTIUM SUMMARY
  54. 54.  LEA1  Grade 3 results to date  Changes in implementations for 2014–2015  LEA 2 Details  Grade 3 results to date  Changes in implementations for 2014–2015 56 2013–2014 RESULTS 2014–2015 PLANS
  55. 55.  Assessment tools aligned with TGRG programs funded by state need approval.  Primary students exhibit high anxiety regarding TGRG, impacting performance and fear of school.  Required testing takes away from instruction time. Embrace testing that collects needed data for all accountability purposes, not just one initiative. 57 TESTING RECOMMENDATIONS
  56. 56.  Continue funding for TGRG development.  Continue monitoring LEA implementation of funded and non-funded TGRG implementation plans, and share “lessons learned.”  Revise RIMP format and collect RIMP data for longitudinal analyses of common deficiencies across the state. 58 TGRG POLICY RECOMMENDATIONS
  57. 57. QUESTIONS suzanne.franco@wright.edu BrumbaughJ@milton-union.k12.oh.us thompsond@piqua.org Making Research Work for Education Extending Your Knowledge Through Resear ch That Works! I Columbus , OH I June 18, 2 0 14
  58. 58. READY OR NOT? E x t e n d i n g Y o u r K n o wl e d g e T h r o u g h R e s e a r c h T h a t W o r k s ! I C o l u m b u s , O H I J u n e 1 8 , 2 0 1 4 Ginny Rammel, Ph.D. Superintendent Milton-Union Exempted Village Schools
  59. 59.  Research and study  Use of multiple forms of data  Create a culture of “calculated risk-takers”  Embed professional development “EVERY STUDENT, EVERY DAY”
  60. 60.  Role model to all at all times  Establish high expectations  Collaborate, share  Trust, be truthful and supportive  Know your staff 63 CULTURE TAKES TIME TO CHANGE!
  61. 61. 64 EXPLORE OPPORTUNITIES  Through grants, pilot studies, action research  Connect with:  Personnel at colleges and universities, OERC  Educators from other districts  Members of professional organizations  Policymakers, legislators
  62. 62.  Milton-Union was involved in a number of grants:  RttT Mini-grant Value-Added  Student Growth Measures  Early Literacy and Reading Readiness  OERC case study The more you and your staff research, study, and share data, the better decisions you make. Collaborating and working together help to create a culture of “Every Student, Every Day.” 65 EMBEDDED PD: BE AN ACTIVE PARTICIPANT
  63. 63. Our culture and the research and data from our grant involvement led to development of our OTES instrument, and a successful year of implementation This trust and openness flowed throughout recent negotiations. 66 RESULTS
  64. 64. All initiatives impact one another and YOU:  OTES  OPES  Graduation requirements  Third Grade Reading Guarantee Keep the main thing the main thing – is what I’m doing going to help students learn? Are we preparing students for “down-the-road” careers? 67 CHANGE WILL OCCUR WITH OR WITHOUT YOU!
  65. 65.  Do the research upfront  Study the data  Reflect, revise if necessary  Building project  Food service program 68 CALCULATED RISK-TAKERS
  66. 66.  All day, every day kindergarten  On-site Head-Start programs  Grouping students by quintiles  H.S. ACT EOC exams  Recognized as a U.S. Department of Education Green Ribbon School  Food service program ended the year in the black! 69 DATA SUPPORTS INITIATIVES
  67. 67.  How do we better prepare students for their futures – colleges, universities, employers?  How can we convey to young parents the importance of their role as a teacher?  How can we differentiate education so all students are better served?  How can we better communicate the results of research and the sharing of data? 70 NEXT STEPS…
  68. 68. QUESTIONS? RAMMELV@MILTON-UNION.K12.OH.US connect@oerc.osu.edu | oerc.osu.edu

×