Your SlideShare is downloading. ×
Oerc 3 27-2014 overall ppt final
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Oerc 3 27-2014 overall ppt final

178
views

Published on

Series Session #3: Research & Measures to Support Quality Early Childhood Initiatives in Ohio (March 27, 2014)

Series Session #3: Research & Measures to Support Quality Early Childhood Initiatives in Ohio (March 27, 2014)

Published in: Education

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
178
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • Note to Debbie: Stephanie and Alicia have their PPT on a jump drive.
  • Program TypeThe types of programs responding to the 2013 survey were similar in distribution to those responding to the 2005 survey. National Association for the Education of Young Children (NAEYC) Accreditation Decrease from 2005 (13.4%) to 2013 (9.6%); which is higher than state average in both 2005 (8%) and 2013 (4%)Highest Rate of Accreditation: Head Start programs (22%)Lowest Rate of Accreditation: ODJFS Faith-Affiliated programs (5%)EnrollmentPercentage of preschoolers, compared to other age groups, was lower in 2013 (52%) than 2005 (65%)Programs with the highest percentage of enrollment for:Infants = ODJFS For-Profit (11%), ODJFS Nonprofit (6%)Toddlers= ODJFS For-Profit (18%), ODJFS Nonprofit (12%)Preschoolers = Head Start (92%), ODJFS Faith-Affiliated (66%)School-Age Children = ODE School-Affiliated (56%), ODE Other (41%)
  • Staffing DiversityMajority still female and Caucasian, although slight increase in racial diversity in 2013Percentage of male directors and teachers increased, particularly among ODE-licensed programs(Directors: ODJFS = 2% to 3%; ODE = 7% to 9%)(Teachers: ODJFS = 3% to 3%; ODE = 5% to 8%)Percentage of directors and teachers identifying as African American, Hispanic/Latino, and Other Ethnicity increased, particularly among ODJFS-licensed programsAfrican American (Directors: ODJFS = 7% to 15%; ODE = 3% to 6%) and (Teachers: ODJFS = 9% to 18%; ODE = 4% to 5%)Hispanic/Latino (Directors: ODJFS = 1% to 1%; ODE = 0% to 1%) and (Teachers: ODJFS = 1% to 3%; ODE = 1% to 1%)Other (Directors: ODJFS = 2% to 2%; ODE = 1% to 2%) and (Teachers: ODJFS = 3% to 4%; ODE = 3% to 3%)In both 2005 and 2013, ODE staff tended to be older than ODJFS staff
  • Overall, percentage of directors with graduate degrees increased, although directors of ODE-licensed programs continued to be more likely to hold graduate degrees 2005: ODJFS = 13%; ODE = 54%2013: ODJFS = 15%; ODE = 62%Overall, percentage of teachers with at least an Associate’s degree increased, although teachers from ODE-licensed programs continued to be more likely hold at least an Associate’s degreeDoes not included assistant teachers2005: ODJFS = 44%; ODE = 80%2013: ODJFS = 57%; ODE = 83%A higher percentage of directors and teachers of ODE-licensed programs have credentials compared to directors and teachers of ODJFS-licensed programs (2013) Directors: ODJFS = 71%; ODE = 106%(2013) Teachers: ODJFS = 29%; ODE = 77%Both directors and teachers of ODE-Affiliated programs hold the highest percentage of EC Teacher Licenses and other types of teaching licensesBoth directors and teachers of Head Start programs hold the highest percentage of Pre-K Associate LicensesDirectors of ODJFS For-Profit programs and teachers of Head Start programs hold the highest percentage of Child Development Associate credentialsBoth directors and teachers of ODE-Affiliated programs hold the highest percentage of EC Teacher Licenses and other types of Teacher LicensesDirectors of ODJFS For-Profit programs hold the highest percentage of CDA Licenses, Head Start directors hold the highest percentage of Pre-K Associate Licenses, and directors of ODE-Affiliated programs hold the highest percentage of EC Teacher and Other Teacher Licenses Head Start teachers hold the highest percentage of CDA and Pre-K Associate Licenses, whereas teachers in ODE-Affiliated programs hold the highest percentage of EC Teacher and Other Teacher Licenses
  • Overall, average salaries increased from 2005 to 2013Only ODE teaching staff had wage increase in line with national averageThere continues to be a large disparity between wages of ODE-licensed programs and ODJFS-licensed programs, ODE-licensed program directors and staff report higher salaries than those from ODJFS-licensed programsCurrent wages for ODE-licensed staff average over $9 more per hour than ODJFS-licensed staffThe highest average wages for ODJFS-licensed teaching staff are lower than the average starting wages for ODE-licensed staff True in both 2005 and 2013ODJFS highest wage: Teachers = $12.58; Assistant Teachers = $9.80ODE lowest starting: Teachers = $13.57; Assistant Teachers = $9.88In general more education was associated with higher hourly payDirectors and staff with graduate degrees in early childhood education or child development claim higher salaries than the same degree in another fieldThe reverse is true for directs and staff with Associate’s or Bachelor’s degrees (Directors an staff with an Associate’s or a Bachelor’s degree in another field earn more than the same degree in ECE or CD)There is a hierarchy in salary which is clearly linked to the program sponsorship for nearly every degree type and credential Directors ― ODE School-Affiliated programsTeachers ― ODE- Licensed programsDIRECTORS: For nearly every degree type and credential, directors of ODE School-Affiliated programs earn the most, whereas ODJFS program directors earn the leastOn average, directors in ODE school affiliated programs earn roughly 80% more than directors in ODJFS for-profit and faith affiliated programs, 70% more than Head Start directors, 67% more than directors in ODJFS nonprofit programs, and 29% more than directors in other programs licensed by ODE. TEACHERS: Teachers from ODE-licensed programs earn the highest wages in every credential category and nearly every degree category (whereas ODJFS program directors earn the least)Teachers in school affiliated and other ODE licensed programs earn about twice as much as teachers from ODJFS licensed programs. In general, ODE licensed program teachers earn 77% to 106% the wage of ODJFS licensed program teachers and 46% to 58% more than Head Start teachers.
  • There is a hierarchy in salary which is clearly linked to the program sponsorship Directors in ODE School-Affiliated programs have highest wages, for nearly every degree type and credential Earn roughly 80% more than directors in ODJFS for-profit and faith affiliated programs, 70% more than Head Start directors, 67% more than directors in ODJFS nonprofit programs, and 29% more than directors in other programs licensed by ODETeachers in ODE- Licensed programs have highest wages, for nearly every degree type and every credentialEarn about twice as much as teachers from ODJFS licensed programs. In general, ODE licensed program teachers earn 77% to 106% the wage of ODJFS licensed program teachers and 46% to 58% more than Head Start teachers
  • Directors in ODE School-Affiliated programs have highest wages, for nearly every degree type and credential Earn roughly 80% more than directors in ODJFS for-profit and faith affiliated programs, 70% more than Head Start directors, 67% more than directors in ODJFS nonprofit programs, and 29% more than directors in other programs licensed by ODE
  • Teachers in ODE- Licensed programs have highest wages, for nearly every degree type and every credentialEarn about twice as much as teachers from ODJFS licensed programs. In general, ODE licensed program teachers earn 77% to 106% the wage of ODJFS licensed program teachers and 46% to 58% more than Head Start teachers
  • Health and Dental Benefits for Full-Time Staff: Percent Offered by Program Type
  • The rates of health, dental, and retirement benefits are:Highest in Head Start programs (health = 92%; dental = 77%; retirement = 93%)Lowest in ODJFS Faith-Affiliated programs (health = 19%; dental = 10%; retirement = 20%)Generally, from 2005 to 2013 the percentage of programs offering benefits other than health and dental coverage has increased (e.g., reduced fee or free child care, paid maternity leave, paid education expenses and training)
  • More on-the-job longevity in 2013 than 2005The percentage of all staff employed at their program for more than 5 years increased 2005 = 32%; 2013 = 43%There is greater longevity for directors and teaching staff in ODE-licensed programs compared to ODJFS-licensed programsDirectors in current position for 3+ years: ODJFS = 70%; ODE = 77%For both 2005 and 2013Over 3 years to 5 years: (2005) ODJFS = 15%; ODE = 18%(2013) ODJFS = 14%; ODE = 20%Over 5 years: (2005) ODJFS = 46%; ODE = 49%(2013) ODJFS = 56%; ODE = 58%Teachers in program for more than 5 years: ODJFS = 45%; ODE = 57%For both 2005 and 2013For both teachers and assistant teachersEmployed for more than 5 years at the program:(2013) Teachers: ODJFS = 45%; ODE = 57%(2013) Assistant Teachers: ODJFS = 31%; ODE = 46%Turnover rates among teaching staff are higher for ODJFS-licensed programs than ODE-licensed programs, and almost identical those of 2005 (and generally higher for assistant teachers than teachers) Turnover = left position in last 12 months(2013) Teachers: ODJFS = 22%; ODE = 11%(2005) Teachers: ODJFS = 21%; ODE = 10%(2013) Assistant Teachers: ODJFS = 30%; ODE = 13%(2005) Assistant Teachers: ODJFS = 29%; ODE = 12%Turnover rates for directors of ODE-licensed and ODJFS-licensed programs continue to be similar, decreasing from 2005 to 2013(2005): ODJFS = 11%; ODE = 10%(2013): ODJFS = 6%; ODE = 6%
  • Most frequently mentioned incentives for staying in ECE/CD field:Better pay: Directors = 53%; Teachers = 68%Better benefits: Directors = 16%; Teachers = 20%Director perceptions of why teachers leaveMost common responses: want/need better pay (55%), schedule/work hours (18%), and job stress/burnout (12%)
  • Note Bullet 1: 19% of children birth-2 years are served in licensed setting.47% of children ages 2-5 are served in licensed settings. Note Bullet 2: Existing network could serve over 300,000 children.
  • Policy Practice is systemic need to provide professional development for programs administered across these 5 agencies. Through early learning Developing a system of PD and our evaluation will determine the extent to which the CPDS has accomplished its stated goals.
  • Designed to improve Ohio’s network of early childhood education programs that are funded through the state.
  • OERC is connected to the CPDS through the second guiding principle, evaluation
  • Second bullet is the focus of our research
  • What was the policy/practice question that initiated this study?
  • Full Question: Appraise the development and usefulness of the 16 new PD modules and the different formats being implemented through the grant. 16 modules addressing a variety of needs, delivered in multiple formats (online and in-person) . Accommodates for different levels of preparation, different context of interaction with children.
  • Appraise the actual delivery of PD content within the established comprehensive PD system (CPDS) that includes: participation, mechanisms and quality of delivery, fidelity of implementation and effects of PD.
  • Guiding Question: Have there been contextual issues that have had an impact upon how the study has progressed (e.g. change in statute, change in program leadership, change in program funding), if so, how has the team dealt with that issue?
  • Guiding Question:What is on the horizon for this research-practice partnership? Is there a natural “next step” for this research? What sorts of changes in policy or practice might occur?
  • Transcript

    • 1. EXAMINING OHIO’S APPROACH TO MEASURING STUDENT SUCCESS SERIES SESSION #3: RESEARCH & MEASURES TO SUPPORT QUALITY EARLY CHILDHOOD INITIATIVES IN OHIO March 27, 2014 Making Research Work for Education
    • 2. 2 WELCOME
    • 3. THE OHIO EDUCATION RESEARCH CENTER A collaborative of Ohio-based research universities & institutions Focused on a statewide research agenda Addressing critical issues of education practice and policy
    • 4.  The mission of the Ohio Education Research Center (OERC) is to develop and implement a statewide, preschool-through- workforce research agenda to address critical issues of education practice and policy. Our intent is to identify and share successful practices; respond to the needs of Ohio’s educators and policymakers; and signal emerging trends. We intend to communicate findings broadly through multiple platforms and networks, producing materials, products and tools to improve educational practice, policy, and outcomes. 4 OERC’S MISSION
    • 5.  State Landscape: Ohio’s Early Learning System  OERC Early Childhood Research  Feedback Activity  OERC Learning Network — Brief Demonstration  Next Steps/Closing 5 TODAY’S AGENDA
    • 6. STATE LANDSCAPE: OHIO’S EARLY LEARNING SYSTEM Stephanie Siddens, Ph.D., Director, Office of Early Learning and School Readiness, Ohio Department of Education Alicia Leatherman, Deputy Director, Division of Child Care, Office of Family Assistance, Ohio Department of Job and Family Services
    • 7. 7 QUESTIONS?
    • 8. OERC EARLY CHILDHOOD RESEARCH Early Childhood Research & Measures Symposium 3/27/2014
    • 9.  Tina Kassebaum, Ph.D., Strategic Research Group (SRG)  Lauren Porter, Program Manager, Ohio Education Research Center at The Ohio State University  Rob Fischer, Ph.D., Research Associate Professor, Jack, Joseph and Morton Mandel School of Applied Social Sciences of Case Western Reserve University  Jennifer Zimmerman, Evaluation Consultant, Resilient Children Project  Jerry M. Jordan, Ph.D., Research Associate, University of Cincinnati’s Evaluation Service Center 9 OERC EARLY CHILDHOOD RESEARCH PRESENTERS
    • 10. OERC Early Childhood Research & Measures Symposium March 27, 2014 2013 WORKFORCE STUDY: OHIO EARLY LEARNING & DEVELOPMENT PROGRAMS
    • 11.  Purpose: To provide an overview of the current characteristics of the childhood workforce in Ohio.  Policy Implication: Over 20 years of research has consistently found a link between the level of education and compensation of the early childhood workforce, and the program quality and outcomes for children.  Key components of study:  Demographics of the childhood workforce  Staff education, credentials, and wage  Benefits offered to staff, turnover, and reasons for staying/leaving 11 PURPOSE OF STUDY
    • 12. History (Similar Studies) 2001 mail survey - 314 ODJFS programs 2005 mail survey - 989 ODJFS & ODE programs 2013: Sample of 3,600 randomly selected early learning and development programs 2,388 ODJFS-licensed programs 1,212 ODE-licensed programs 12 STUDY BACKGROUND
    • 13. Surveys: Program Director Survey Teacher Survey Data Collection: Web-based surveys Invitation letters mailed to Program Directors  Two mailings 2 weeks apart Telephone and (if possible) email follow-up invitations RESEARCH DESIGN 4
    • 14. Response Rate: 30% for directors and 23% for selected teachers 40% for ODJFS and 20% for ODE 10% held NAEYC Accreditation Two Reports Generated General Analysis (comparisons to 2005 study) A Profession Divided (comparisons by program type) RESPONSE RATES & REPORTS 5
    • 15.  Six program sponsorship types  Total Programs = 1060 PROGRAM CHARACTERISTICS: SPONSORSHIP TYPES 10.1% 7.5% 32.1% 20.8% 20.5% 9.0% ODE School-Affiliated ODE Other ODJFS For-Profit ODJFS Nonprofit ODJFS Faith-Affiliated Head Start 6
    • 16.  Majority still female and Caucasian  Slight increase in gender and ethnic diversity in 2013  In both 2005 and 2013, ODE staff tended to be older than ODJFS staff WORKFORCE DEMOGRAPHIC DIVERSITY 2005 2013 ODJFS ODE ODJFS ODE Caucasian Director 91% 95% 82% 91% Teacher 87% 92% 76% 91% Female Director 98% 94% 97% 91% Teacher 97% 96% 97% 92% 7
    • 17. DEGREES HELD BY PROGRAM STAFF 57% 43% 20% 17% 43% 57% 80% 83% ODJFS 2005ODJFS 2013 ODE… ODE… Teacher Less than AA AA or Higher 88% 85% 46% 38% 12% 15% 54% 62% ODJFS 2005ODJFS 2013 ODE… ODE… Director Less than Graduate Degree Graduate Degree 8
    • 18.  A higher percentage of directors and teachers of ODE- licensed programs have credentials compared to those of ODJFS-licensed programs CREDENTIALS HELD BY PROGRAM STAFF Teachers Directors Credential Program with Highest Percentage % Program with Highest Percentage % Child Development Associate Head Start 26.2 ODJFS For-Profit 23.2 Pre-K Associate License Head Start 28.6 Head Start 23.2 EC Teacher License ODE Other 47.0 ODE (both) & Head Start 25.2 - 26.3 Other Teaching License ODE School- Affiliated 54.4 ODE School- Affiliated 72.0 9
    • 19.  Continued disparity between wages of ODE-licensed programs and ODJFS-licensed programs  Current wages for ODE-licensed staff average over $9 more per hour  The average highest wage reported for ODJFS-licensed teaching staff ($12.58) is lower than the average starting wage reported for ODE- licensed staff ($13.57)  In general, more education is associated with higher hourly pay  Directors and staff with graduate degrees in ECE or CD claim higher salaries than the same degree in another field  However, program sponsorship clearly matters as ODE- licensed staff tend to earn higher wages than ODJFS-licensed staff for the same degree WAGES 10
    • 20. AVERAGE CURRENT WAGES REPORTED BY STAFF $13.56 $23.48 $16.08 $25.42 ODJFS ODE Directors 2005 2013 $9.59 $16.32 $11.36 $20.79 ODJFS ODE Teaching Staff 2005 2013 11
    • 21. ODE School- Affiliated ODJFS Faith- Affiliated ODJFS For- Profit ODJFS Nonprofit ODE Other Head Start Credential Child Development Associate $18.58 $15.52 $14.42 $12.99 $12.29 $17.00 Pre-K Associate License $29.00 $14.39 $15.26 $16.24 $23.45 $17.33 EC Teacher License $38.52 $14.57 $16.97 $16.04 $21.78 $15.36 Other Teaching License $31.88 $14.79 $16.48 $18.43 $25.64 $15.94 Highest Degree Earned AA ECE or CD $16.35 $14.74 $14.82 $15.76 $12.89 $17.73 BA/BS ECE or CD $21.99 $15.49 $16.94 $18.22 $19.12 $15.98 Graduate Degree in ECE or CD $36.12 $16.90 $19.00 $21.08 $24.78 $18.18 Graduate Degree in Another Field $31.32 $15.10 $12.46 $19.42 $27.30 $15.07 DIRECTORS: AVERAGE HOURLY WAGE 12
    • 22. TEACHERS: AVERAGE HOURLY WAGE ODE School- Affiliated ODJFS Faith- Affiliated ODJFS For- Profit ODJFS Nonprofit ODE Other Head Start Credential Child Development Associate $15.54 $11.15 $11.14 $11.92 - $13.14 Pre-K Associate License $21.00 $11.74 $10.43 $16.65 $20.97 $15.53 EC Teacher License $24.62 $13.07 $11.61 $11.77 $25.10 $13.43 Other Teaching License $23.24 $12.87 $11.92 $12.32 $25.80 $13.21 Highest Degree Earned AA ECE or CD $15.94 $12.44 $10.64 $12.29 $14.23 $14.28 BA/BS ECE or CD $16.92 $12.29 $12.32 $12.10 $19.66 $14.44 Graduate Degree in ECE or CD $26.84 $14.22 $12.00 $13.90 $31.75 $15.03 Graduate Degree in Another Field $25.47 $10.86 $10.70 $10.99 $20.79 - 13
    • 23. BENEFITS BY PROGRAM TYPE 62% 19% 33% 38% 71% 92% 49% 10% 20% 28% 53% 77% 81% 20% 24% 51% 71% 93% ODE School- Affiliated ODJFS Faith- Affiliated ODJFS For- Profit ODJFS Nonprofit ODE Other Head Start Health Benefits Dental Benefits Retirement Benefits 14
    • 24. Generally, from 2005 to 2013 the percentage of programs offering benefits other than health and dental coverage has increased (e.g., reduced fee or free child care, paid maternity leave, paid education expenses and training) OTHER EMPLOYEE BENEFITS 15
    • 25.  Increase from 2005 to 2013 in percentage of staff employed at their program for more than 5 years  Greater longevity for directors and teaching staff in ODE- licensed programs compared to ODJFS-licensed programs  Turnover rates among teaching staff are higher for ODJFS- licensed programs than ODE-licensed programs, and almost identical to those of 2005  Turnover rates for directors of ODE-licensed and ODJFS- licensed programs continue to be similar, decreasing from 2005 to 2013 LONGEVITY & TURNOVER 16
    • 26. TURNOVER RATES 11% 21% 29% 10% 10% 12% Director Teachers Assistant Teachers 2005 6% 22% 30% 6% 11% 13% Director Teachers Assistant Teachers 2013 ODJFS ODE 17
    • 27. ODE School- Affiliated ODJFS Faith- Affiliated ODJFS For- Profit ODJFS Nonprofit ODE Other Head Start Total Directors Better pay 26% 56% 61% 48% 52% 76% 53% Better benefits 3% 18% 23% 21% 12% 0% 16% More help/support/ resources 3% 6% 15% 15% 8% 19% 11% Work more/fewer hours 7% 9% 10% 10% 4% 0% 8% More staff/retain good, qualified staff 3% 4% 13% 10% 8% 5% 8% Teachers Better pay 24% 70% 69% 78% 55% 73% 68% Better benefits 12% 28% 18% 26% 10% 9% 20% Full-time employment or better hours 29% 12% 5% 9% 5% 9% 9% Stable enrollment/job security 12% 6% 0% 3% 15% 23% 6% MOST FREQUENTLY MENTIONED INCENTIVES FOR STAYING IN ECE/CD FIELD 18
    • 28.  Results highlight the continued disparity between ODE-licensed and ODJFS-licensed programs  Compared to staff of ODJFS-licensed programs, ODE-licensed program staff tend to:  Receive higher wages and be offered more benefits  Be older, more gender diverse, and less ethnically diverse  Attain more credentials and higher levels of education  Turnover less frequently and be employed longer  Despite increases in diversity, wages, and non-medical benefits, the gap in education and wages between ODE-licensed and ODJFS-licensed programs persists  Teachers and directors holding the same degrees and credentials are paid more than two times the rate in a school-affiliated setting than in for-profit centers licensed by ODJFS. FINDINGS SUMMARY 19
    • 29. THANK YOU connect@oerc.osu.edu | oerc.osu.edu
    • 30. COMPREHENSIVE PROFESSIONAL DEVELOPMENT SYSTEM EVALUATION Lauren Porter, Ohio Education Research Center Debbie Zorn, University of Cincinnati Imelda Castañeda-Emenaker, University of Cincinnati
    • 31.  There are 860,000 children ages 5 and under.  Currently over 11,000 licensed early learning programs. 3/27/2014Early Childhood Research & Measures Symposium 31 Existing Landscape as of February 2013:
    • 32.  Ohio Department of Education  Ohio Department of Job and Family Services  Ohio Department of Health  Ohio Department of Mental Health  Ohio Department of Developmental Disabilities 3/27/2014Early Childhood Research & Measures Symposium 32 Agencies Serving Children Birth–5
    • 33.  RttT ELC Grant – 2011.  Designed to improve state funded network of programs.  One ELC initiative: develop extensive early learning professional development system. 3/27/2014Early Childhood Research & Measures Symposium 33 Early Learning Challenge (ELC) Grant
    • 34.  Mission: CPDS designed to provide appropriate professional development to early learning and development professionals working with children aged birth-5 years, enhancing their competencies and skills. 3/27/2014Early Childhood Research & Measures Symposium 34 Comprehensive Professional Development System (CPDS)
    • 35.  Monitored  Evaluated  Aligned to standards from multiple sectors  Based on research  Supply data for analysis, reporting, and evaluation  Support RttT-ELC Outcomes  Coordinated, using common elements when possible  Aligned to Ohio Core Knowledge and Competencies 3/27/2014Early Childhood Research & Measures Symposium 35 CPDS Guiding Principles
    • 36. Establishment of a seamless Comprehensive Professional Development System.  Activities include interviews of policy stakeholders & practitioner stakeholders and survey of the Regional PD Network. 3/27/2014Early Childhood Research & Measures Symposium 36 Evaluation Question 1
    • 37. Appraise the development and usefulness of the 16 new PD modules.  Activities include a needs assessment conducted by OCCRRA, expert review of modules, and PD participant feedback. 3/27/2014Early Childhood Research & Measures Symposium 37 Evaluation Question 2
    • 38. Appraise the actual delivery of PD content.  Activities include retrospective interviews with PD participants and Regional PD network members and observation of PD modules. 3/27/2014Early Childhood Research & Measures Symposium 38 Evaluation Question 3
    • 39.  OCCRRA Project Team and the OERC team have met regularly to gather materials related to the evaluation and train on evaluation instruments.  Integral to the CPDS evaluation are the interviews from a variety of different practitioners:  OCCRRA Project Team  Regional PD Coordinators  PD participants 3/27/2014Early Childhood Research & Measures Symposium 39 Researcher-Practitioner Collaboration
    • 40.  Currently in preparation stages of research.  Institutional Review Board (IRB) approval is being sought through The Ohio State University.  While this application is pending, research activities (interviews, surveys, data analysis) may not be conducted.  Delay has presented an opportunity to develop materials for later stages of the research project. 3/27/2014Early Childhood Research & Measures Symposium 40 Boon and Bane of IRB
    • 41.  Upon IRB approval, next steps are: initial data access, interviews, and an in-depth examination of one module from development to implementation and impacts.  Potential policy impacts: PD options that better serve early learning and development professionals in all settings and a more aligned approach to early learning professional development across state agencies. 3/27/2014Early Childhood Research & Measures Symposium 41 Moving Forward
    • 42. THANK YOU
    • 43. Robert L. Fischer, Ph.D., Claudia J. Coulton, Ph.D., and Seok-Joo Kim, Ph.D. Center on Urban Poverty & Community Development Jack, Joseph and Morton Mandel School of Applied Social Sciences Case Western Reserve University, Cleveland, Ohio March 27, 2014 – Columbus, OH INVESTIGATING THE PATHWAY TO PROFICIENCY FROM BIRTH THROUGH 3RD GRADE: INTEGRATED DATA SYSTEM (IDS) APPROACH
    • 44. BACKGROUND 44 Study significance: • Importance of early childhood exposures o Early exposure to stressful circumstances, environmental hazards, and less than optimal early learning environments negatively and persistently affect early development. • Usefulness of longitudinal data • State adopted ―3rd Grade Reading Guarantee‖ to ensure that students pass reading proficiency test before advancing beyond 3rd grade • Districts can be more aware of risk factors for students being held back as the policy is implemented
    • 45. BACKGROUND 45 Study aims: Aim 1. Assess the practicality of linking early childhood and K-3 student records and potential usefulness of the resulting information to local schools(A)&(B) Aim 2. Determine how individual, family, and environmental risk factors in early childhood interact with participation in early childhood education programs to influence kindergarten readiness(A) Aim 3. Estimate the effects of early childhood risk factors and experiences on student progress over grades 1 to 3(B) Aim 4. Identify child-level indices, including kindergarten readiness and reading-growth trajectories, that in their combination accurately predict reading proficiency in third grade(A)&(B) Note: (A) In process and (B) After April, 2014
    • 46. COHORT DESIGN 46 Cohort 1 Cohort 2 Cohort 3 Cohort 4 B 3rdK B 3rdK B 3rdK B 3rdK 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013Year Retrospective Prospective
    • 47. SAMPLING 47 Criteria Enrolled in Cleveland Metropolitan School District (CMSD) Kindergarten in the school years of 2008- 2011 First time enrollment of Kindergarten Have a valid KRA-L score With Ohio Birth-Certificate & Addresses 16,840 15,581 11,999 9,777 Selection criteria N • Unduplicated cases (students) • Will be updated from EMIS data • Will be updated from Medicaid data • Missing imputation
    • 48. KRA-L & 3rd grade reading* Neighborhood • Concentrated disadvantage • Immigrant concentration* • Crime * School • School characteristics * Service • Home visiting • Head Start • Preschool • Universal Pre-K CONCEPTUAL MODEL 48 Child • Demographic • Low birth weight • Age at kindergarten • Disability Family • TANF/SNAP/ Medicaid • Mother’s edu. • Teen mother • Maltreatment • Foster care * Currently not available; will be included
    • 49. Educational Outcomes • KRA-L score • 3rd grade reading proficiency • Attendance Child Context • Demographic • Low birth weight • Age at kindergarten • Disability Family Context • TANF/SNAP/ Medicaid • Mother’s education • Teen mother • Child maltreatment • Foster care Service Context • Home visiting • Head Start • Preschool • Universal Pre-K Mobility • School / Residential INTEGRATED DATA SYSTEM 49 CHILD system Educational Outcomes • KRA-L score • 3rd grade reading proficiency • Attendance School Context • School characteristics Mobility • School OLDA Neighborhood context • Concentrated disadvantage • Immigrant concentration • Crime NEO CANDO Data Integration by State Student ID Data Integration By Census tract Data Integration By ECIID
    • 50. CHILD SYSTEM 50 Introduction  Key data system for this study  Data helps inform our understanding of the early childhood system  Individuals and families interact with multiple systems and services, so integrated data offers a more complete view of reality [―Big Data‖]  Understanding of how systems work and how to better meet existing needs can be informed by integrated data  Service models emphasize long term and collective impact, so data needed across services and over time
    • 51. ID6 ID5 ID4 ID3 ID2 ID1 • Abuse/neglect reports* • Foster care* • Home visiting* • Special needs child care* • Early childhood mental health • Universal pre-k* • Attendance* • KRA-L* • Proficiency test* • Graduation test • Disability* • Medicaid* • Food Stamp* • TANF* • Child care voucher* • Infant mortality • Elevated Blood Lead • Teen births* • Low weight birth* CHILD SYSTEM Concept Public Assist Public School Common ID Childhood Integrated Longitudinal Data (CHILD) System 51 *: Data for this project
    • 52. CHILD SYSTEM Structure Geocode & Standardize Updated IDS Register-includes ID#s, names, addresses, DOB, etc. IDS Register- includes ID#s, names, addresses, DOB, etc. Outcomes E.g. Kindergarten Readiness Scores among children in UPK program Profiles E.g. Birth characteristics & service used for children entering kindergarten Geographic E.g. % LBW births receiving ongoing home visits by neighborhood Time Trends E.g. Total Children Served by birth cohort Data files-Births, Home Visiting, DCFS, UPK, KRA-L, Medicaid, etc. Longitudinal Master Files for Each Data Source REPORTS Match New Records to IDS Register 52
    • 53. NEIGHBORHOOD MEASURE 53 Principal Component Analysis Poverty Female- headed families African- American Un- employ- ment Welfare recipient .9 .8 .8 .5 .7 N of neighborhoods=501 census tracts of Cuyahoga County; American Community Survey 2009 Concentrated Disadvantage
    • 54. DESCRIPTIVE ANALYSIS KRA-L (N=9777) % Mean(SD) p1) Child Context Male 50.8 15.2(6.9) 0.000 Female 49.2 16.7(6.8) Low birth weight (No) 88.0 16.1(6.9) 0.001 Low birth weight (Yes) 12.0 15.2(7.0) Race White 17.2 16.5 (7.1) 0.000 2) Others 4.6 16.7(7.0) Hispanic 9.9 13.7(6.8) African-American 68.3 16.1(6.8) Age at Kindergarten (Months) 64.7 (4.2) 3) 0.144) 0.000 Without disability at kindergarten 96.9 16.1(6.8) 0.000 Disability at kindergarten 3.1 11.5(6.3) Family Context Born to teenage mother (No) 76.1 16.1(6.9) 0.000 Born to teenage mother (Yes) 23.9 15.5(6.6) Born to mother without HS degree 44.2 15.0(6.6) 0.000 Born to mother with HS degree 55.8 16.8(7.0) Months of <150% of poverty line from birth to kindergarten3) 43.2 (23.2) 3) -.0.144) 0.000 Substantiated/indicated child abuse before Kindergarten (No) 86.2 16.1(6.9) 0.000 Substantiated/indicated child abuse before Kindergarten (Yes) 13.8 14.9(6.6) Foster care placement before age 5 (No) 94.2 16.3(6.8) 0.025 Foster care placement before age 5 (Yes) 5.8 15.3(6.6) KRA-L score 16.0(6.9) Note: 1) p form t-test, 2) p form One-way ANOVA, 3) Mean (SD), 4) Pearson-r and p 54
    • 55. DESCRIPTIVE ANALYSIS KRA-L (N=9777) % Mean(SD) p1) Service Context Early intervention ever (No) 88.9 16.2(6.8) 0.000 Early intervention ever (Yes) 11.1 13.8(6.7) Number of ongoing home visiting over 12 times (No) 77.6 16.2(6.9) 0.000 Number of ongoing home visiting over 12 times (Yes) 22.5 15.1(6.7) Welcome home visiting ever (No) 68.9 15.8(6.8) 0.000 Welcome home visiting ever (Yes) 31.1 16.4(6.9) Headstart over 6 months (No) 90.6 15.8(6.7) 0.000 Headstart over 6 months (Yes) 9.4 17.7(6.7) CMSD Preschool over 120 days (No) 81.3 15.3(6.7) 0.000 CMSD Preschool over 120 days (Yes) 18.7 18.7(7.1) Universal Pre-K ever over 6 months (No) 97.7 15.9(6.9) 0.000 Universal Pre-K ever over 6 months (Yes) 2.3 18.8(6.6) Neighborhood context Concentrated disadvantages in 2009 by Census tract (M=0, SD=1) 0.8 (0.9)2) -.0.043) 0.000 KRA-L score 16.0(6.9) Note: 1) p form t-test, 2) Mean (SD), 3) Pearson-r and p 55
    • 56. OLS REGRESSION KRA-L score (Dependent variable) B SE t p B Intercept -1.261 1.011 -1.25 .213 Child Context Gender (Female=1) 1.381 0.128 10.78 0.000 0.100 Low birth weight (Yes=1) -0.644 .0202 -3.19 .001 -0.030 Race: Reference (White and others, Yes=1) Hispanic (Yes=1) -2.515 .0245 -10.25 0.000 -0.109 African-American (Yes=1) 0.114 0.183 0.62 0.533 0.008 Age at Kindergarten (in months) 0.267 0.015 17.38 0.000 0.163 Disability at Kindergarten (Yes=1) -6.354 0.387 -16.41 0.000 -0.161 Family Context Born to teenage mother (Yes=1) -0.194 0.185 -1.05 0.295 -0.012 Born to mother without HS degree (Yes=1) -1.136 0.141 8.04 0.000 -0.082 Months of <150% of poverty line from birth to kindergarten (months) -0.037 0.003 -11.89 0.000 -0.124 Substantiated/indicated child abuse before Kindergarten (Yes=1) -0.698 0.205 -3.40 0.001 -0.035 Foster care placement before Kindergarten (Yes=1) 0.714 0.306 2.34 0.020 0.024 Service Context: Before kindergarten Early intervention ever (Yes=1) -1.935 0.221 -8.77 0.000 -0.088 Ongoing home visiting over 12 times (Yes=1) -0.365 0.160 -2.28 0.023 -0.022 Welcome home visiting ever (Yes=1) 0.917 0.163 5.62 0.000 0.062 Head Start over 6 months (Yes=1) 1.673 0.222 7.55 0.000 0.071 CMSD Preschool over 120 days (Yes=1) 4.003 0.171 23.35 0.000 0.227 Universal Pre-K ever over 6 months (Yes=1) 1.919 0.376 5.10 0.000 0.047 Neighborhood Context Concentrated disadvantages in 2009 by Census tract -0.239 0.085 -2.80 0.005 -0.031 Model Fit: F(18, 9752)=105.04, p<.001, R2 =16.24% | N=9777 56
    • 57. KRA-L Band and poverty rate in Cleveland Metropolitan School District, OH Source: 1. Cleveland Metropolitan School District (CMSD) data 2. American Community Survey 2009 (www.census.gov) Note: Kindergartners in the school years of 2008-2011 (N=9777)
    • 58. IMPLICATIONS 58  Collaboration with Cleveland Metropolitan School District (CMSD) and Early childhood agencies o Data Sharing o Uses - Building profiles - Community collaborative planning - Risk factor reduction  Helpful to inform educational planning; especially schools with large numbers of disadvantaged students  Understand challenges for kindergarten readiness and 3rd grade guarantee
    • 59. FUTURE PLAN 59 Data work  Check duplicated cases again*  Prepare for variables between K-3rd grade*  Update missing data from Education Management Information System (EMIS; Expected: April, 2014)**  Adding more level-2 data o Neighborhood variables from NEO CANDO* o School characteristics from EMIS***  Dealing with missing data o Missing imputation** Note: *Completed, **Expected after April, 2014, ***Planned
    • 60. FUTURE PLAN 60 Analysis  Extended model o Outcome variable: 3rd grade reading proficiency  Use multi-level analyses o School-level o Census tract-level  Geo-analysis / mobility analysis o Spatial auto-correlation o School or residential mobility
    • 61. Thank you! 61 Contact Information: Robert L. Fischer, Ph.D. (fischer@case.edu) Resources  Center on Urban Poverty & Community Development: http://povertycenter.case.edu/  Ohio Education Research Center: http://oerc.osu.edu/  NEO CANDO: http://neocando.case.edu/
    • 62. RESILIENT CHILDREN PROJECT (RCP)
    • 63.  Funded in February 2011 by the Cincinnati/ Northern Kentucky Social Innovation Fund  Expands the scope of two existing Early Childhood Mental Health Consultation (ECMHC) projects  Provides comprehensive services to support young children’s social and emotional development  Increases the capacity of early care and education administrators and caregivers Resilient Children Project
    • 64.  To what extent, if any, does the programmatic delivery of ECMHC increase the kindergarten readiness of the preschool children who receive the ECMHC?  To what extent, if any, do the adult professionals participating in the programmatic delivery of ECMHC increase their level of self efficacy in the delivery of ECMCH? RCP Research Questions
    • 65. Year 1:  Intervention Group: Early childhood programs already participating in ECMHC  Comparison Group: Early childhood programs on a ―wait list‖ for ECMHC services Year 2: All sites receive ECMHC services  Permits examination of effects of implementation of ECMHC services in new sites  Comparison of new and long-term sites RCP Evaluation Design
    • 66.  Linking preschool assessment data to demographics and kindergarten entry data  Measuring teacher stress and self efficacy  New measurement strategies for teacher knowledge and skill  Clarification of key program components and their impact  Exploration of program climate The Evolution of an Evaluation
    • 67.  Child, teacher, and staff turnover/mobility  Differences in program models  ―Dosage‖ complications  Changes at the state  Linkage to kindergarten assessment data sets Life Happens….
    • 68.  Linking program outcomes  Revising process monitoring tools  Enhancing early childhood data systems  Continued exploration of program climate  Where do families play into the equation? What’s Next
    • 69. Thank You
    • 70. 70 QUESTIONS FOR PANELISTS?
    • 71. 71 FEEDBACK ACTIVITY
    • 72.  Erin Joyce, Director, Battelle for Kids, and co-chair, OERC Outreach Committee 72 OERC LEARNING NETWORK
    • 73. June 18, 2014 — Half-Day Learning Session Columbus Convention Center Mid-September — Full-Day Learning Session Columbus 73 OERC NEXT STEPS
    • 74. THANK YOU! connect@oerc.osu.edu | oerc.osu.edu