Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborative Evaluation

Checking Assumptions to Accomplish Collaborative Evaluation National Partnership for Educational Access 2 nd Annual Conference April 8, 2010

A little about us… Carl Ackerman Clarence T. C. Ching PUEO (Partnerships in Unlimited Education) Program Brenda McLaughlin National Summer Learning Association Beth Casey Middle Grades Partnership

A little about you…
What organization do you represent?
What do you hope to learn?

What do kids stand to lose?
Academic knowledge
Healthy habits
Access to meals
Technology know-how

What do kids stand to lose? Academic Knowledge
Since 1906, numerous studies have confirmed that children experience learning losses in math and reading without continued opportunities for regular practice (White, Heyns, Cooper, Alexander)
Disadvantaged youth are disproportionately affected by losses in literacy skills

What do kids stand to lose? Academic Knowledge
Two-thirds of the ninth grade reading achievement gap can be explained by unequal access to summer learning opportunities, contributing to fewer disadvantaged youth graduating from high school or entering college (Alexander, Entwisle & Olson, 2007)

Sources: Doris Entwisle, Karl Alexander, and Linda Olson, Children, Schools, and Inequality , 1997, Table 3.1 Disadvantaged, by Year Better-Off, by Year -10 40 90 140 190 1 2 3 4 5 -10 40 90 140 190 1 2 3 4 5 Disadvantaged, by Year Better-Off, by Year -10 40 90 140 190 1 2 3 4 -10 40 90 140 190 1 2 3 4 SCHOOL YEAR CUMULATIVE GAINS SUMMER CUMULATIVE GAINS

What do kids stand to lose? Healthy Habits
Children gain BMI nearly twice as fast during the summer as during the school year (von Hippel, Powell, Downey & Rowland, 2007)
Black and Hispanic children, and children who are already overweight, experience healthier BMI gain during the school year
School-based fitness interventions can promote better health, but without sustained intervention these benefits are lost over the summer break (Carrel et al., 2007)

What do kids stand to lose? Access to Meals
In July 2008, 17.3 children received Summer Nutrition for every 100 low-income students who received lunch in the 2007-2008 school year (FRAC)
Where you live makes a difference!
Low of 4.4% in Mississippi to a high of 88.8% in Washington DC
Only 10 states manage to serve 25% of their kids
11 states serve less than 10%

What do kids stand to lose? Technology Know-How
Library and technology usage differs by income (Neumann & Celano, 2008)
Print materials in the library:
Lower-income children choose less challenging material , with less print and lower reading levels
Read 1 line of print for every 3 read by middle income children
Spent less time with each book – 6.6 versus 12 minutes

Evaluation Considerations
How is our population the same as or different from the research? (demographics)
How is our program being implemented? (process)
Is our program being implemented as we intend it to? (fidelity to implementation)
What parts of our program are influencing our results? (quality)
Are we making a difference? (impact)

MGP Background
Pilot summer – 2005
500 students annually
13 sites involving 20 schools and three universities
This program serves children entering grades: 7 th , 8 th , 9 th and 10 th

MGP Background
Student/staff ratio: 1 to 6
Urban
Baltimore, Maryland
162 hours per summer

MGP Background
2008 budget: $1.6 million
100% foundation or private individual funded in 2008

PUEO Background
Started in 2005
Served 160 participants in 2009. Anticipate serving 200 in 2010.
Operates a single site. Older participants may meet at a different site.
6 th through 12 th

PUEO Background
Student/staff ratio varies by age group. 1:5 for younger children to 1:13 for oldest
Participants from both urban and rural backgrounds
Honolulu, Hawai’i at Punahou School
100+ hours of programming, depends upon level

PUEO Background
Participants are all from low-income families. Some from immigrant households.
Total Budget: $411,906
Supported through in-kind support from Punahou School and foundations

Key Evaluation Documents Logic Model & Evaluation Questions
Why would a program use this document? For what purpose?
What would evaluators need to know to write this document?
Which program staff would be involved? When? How long?

Partnering with Evaluators Two Approaches
Assess & Validate
Usually random assignment or quasi-experimental
Collaborative Stakeholder Approach
Program development focused
Process model
Not mutually exclusive
Be an informed consumer!

Contact Us Carl Ackerman [email_address] Beth Casey [email_address] Brenda McLaughlin [email_address]

Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborative Evaluation

by Marissa Lowman on Apr 13, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 2

Statistics

Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborative Evaluation — Presentation Transcript