Checking Assumptions to Accomplish Collaborative Evaluation National Partnership for Educational Access 2 nd  Annual Confe...
A little about us… Carl Ackerman Clarence T. C. Ching PUEO (Partnerships in Unlimited Education) Program Brenda McLaughlin...
A little about you… <ul><li>What organization do you represent? </li></ul><ul><li>What do you hope to learn? </li></ul>
What do kids stand to lose? <ul><li>Academic knowledge </li></ul><ul><li>Healthy habits </li></ul><ul><li>Access to meals ...
What do kids stand to lose? Academic Knowledge <ul><li>Since 1906, numerous studies have confirmed that children experienc...
What do kids stand to lose? Academic Knowledge <ul><li>Two-thirds of the ninth grade reading achievement gap can be explai...
Sources:  Doris Entwisle, Karl Alexander, and Linda Olson,  Children, Schools, and Inequality , 1997, Table 3.1 Disadvanta...
What do kids stand to lose? Healthy Habits <ul><li>Children gain BMI nearly twice as fast during the summer as during the ...
What do kids stand to lose? Access to Meals <ul><li>In July 2008, 17.3 children received Summer Nutrition for every 100 lo...
What do kids stand to lose? Technology Know-How <ul><li>Library and technology usage differs by income (Neumann & Celano, ...
Evaluation Considerations <ul><li>How is our population the same as or different from the research? (demographics) </li></...
MGP Background <ul><li>Pilot summer – 2005 </li></ul><ul><li>500 students annually </li></ul><ul><li>13 sites involving 20...
MGP Background <ul><li>Student/staff ratio: 1 to 6 </li></ul><ul><li>Urban </li></ul><ul><li>Baltimore, Maryland </li></ul...
MGP Background <ul><li>2008 budget: $1.6 million </li></ul><ul><li>100% foundation or private individual funded in 2008 </...
PUEO Background <ul><li>Started in 2005 </li></ul><ul><li>Served 160 participants in 2009. Anticipate serving 200 in 2010....
PUEO Background <ul><li>Student/staff ratio varies by age group. 1:5 for younger children to 1:13 for oldest </li></ul><ul...
PUEO Background <ul><li>Participants are all from low-income families. Some from immigrant households.  </li></ul><ul><li>...
Key Evaluation Documents Logic Model & Evaluation Questions <ul><li>Why would a program use this document? For what purpos...
Partnering with Evaluators Two Approaches <ul><li>Assess & Validate </li></ul><ul><ul><ul><li>Usually random assignment or...
Contact Us Carl Ackerman [email_address] Beth Casey [email_address] Brenda McLaughlin [email_address]
Upcoming SlideShare
Loading in …5
×

Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborative Evaluation

696 views

Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

Research Policy & Evaluation: Checking Assumptions to Accomplish Collaborative Evaluation

  1. 1. Checking Assumptions to Accomplish Collaborative Evaluation National Partnership for Educational Access 2 nd Annual Conference April 8, 2010
  2. 2. A little about us… Carl Ackerman Clarence T. C. Ching PUEO (Partnerships in Unlimited Education) Program Brenda McLaughlin National Summer Learning Association Beth Casey Middle Grades Partnership
  3. 3. A little about you… <ul><li>What organization do you represent? </li></ul><ul><li>What do you hope to learn? </li></ul>
  4. 4. What do kids stand to lose? <ul><li>Academic knowledge </li></ul><ul><li>Healthy habits </li></ul><ul><li>Access to meals </li></ul><ul><li>Technology know-how </li></ul>
  5. 5. What do kids stand to lose? Academic Knowledge <ul><li>Since 1906, numerous studies have confirmed that children experience learning losses in math and reading without continued opportunities for regular practice (White, Heyns, Cooper, Alexander) </li></ul><ul><li>Disadvantaged youth are disproportionately affected by losses in literacy skills </li></ul>
  6. 6. What do kids stand to lose? Academic Knowledge <ul><li>Two-thirds of the ninth grade reading achievement gap can be explained by unequal access to summer learning opportunities, contributing to fewer disadvantaged youth graduating from high school or entering college (Alexander, Entwisle & Olson, 2007) </li></ul>
  7. 7. Sources: Doris Entwisle, Karl Alexander, and Linda Olson, Children, Schools, and Inequality , 1997, Table 3.1 Disadvantaged, by Year Better-Off, by Year -10 40 90 140 190 1 2 3 4 5 -10 40 90 140 190 1 2 3 4 5 Disadvantaged, by Year Better-Off, by Year -10 40 90 140 190 1 2 3 4 -10 40 90 140 190 1 2 3 4 SCHOOL YEAR CUMULATIVE GAINS SUMMER CUMULATIVE GAINS
  8. 8. What do kids stand to lose? Healthy Habits <ul><li>Children gain BMI nearly twice as fast during the summer as during the school year (von Hippel, Powell, Downey & Rowland, 2007) </li></ul><ul><li>Black and Hispanic children, and children who are already overweight, experience healthier BMI gain during the school year </li></ul><ul><li>School-based fitness interventions can promote better health, but without sustained intervention these benefits are lost over the summer break (Carrel et al., 2007) </li></ul>
  9. 9. What do kids stand to lose? Access to Meals <ul><li>In July 2008, 17.3 children received Summer Nutrition for every 100 low-income students who received lunch in the 2007-2008 school year (FRAC) </li></ul><ul><li>Where you live makes a difference! </li></ul><ul><ul><ul><li>Low of 4.4% in Mississippi to a high of 88.8% in Washington DC </li></ul></ul></ul><ul><ul><ul><li>Only 10 states manage to serve 25% of their kids </li></ul></ul></ul><ul><ul><ul><li>11 states serve less than 10% </li></ul></ul></ul>
  10. 10. What do kids stand to lose? Technology Know-How <ul><li>Library and technology usage differs by income (Neumann & Celano, 2008) </li></ul><ul><li>Print materials in the library: </li></ul><ul><ul><ul><li>Lower-income children choose less challenging material , with less print and lower reading levels </li></ul></ul></ul><ul><ul><ul><li>Read 1 line of print for every 3 read by middle income children </li></ul></ul></ul><ul><ul><ul><li>Spent less time with each book – 6.6 versus 12 minutes </li></ul></ul></ul>
  11. 11. Evaluation Considerations <ul><li>How is our population the same as or different from the research? (demographics) </li></ul><ul><li>How is our program being implemented? (process) </li></ul><ul><li>Is our program being implemented as we intend it to? (fidelity to implementation) </li></ul><ul><li>What parts of our program are influencing our results? (quality) </li></ul><ul><li>Are we making a difference? (impact) </li></ul>
  12. 12. MGP Background <ul><li>Pilot summer – 2005 </li></ul><ul><li>500 students annually </li></ul><ul><li>13 sites involving 20 schools and three universities </li></ul><ul><li>This program serves children entering grades: 7 th , 8 th , 9 th and 10 th </li></ul>
  13. 13. MGP Background <ul><li>Student/staff ratio: 1 to 6 </li></ul><ul><li>Urban </li></ul><ul><li>Baltimore, Maryland </li></ul><ul><li>162 hours per summer </li></ul>
  14. 14. MGP Background <ul><li>2008 budget: $1.6 million </li></ul><ul><li>100% foundation or private individual funded in 2008 </li></ul>
  15. 15. PUEO Background <ul><li>Started in 2005 </li></ul><ul><li>Served 160 participants in 2009. Anticipate serving 200 in 2010. </li></ul><ul><li>Operates a single site. Older participants may meet at a different site. </li></ul><ul><li>6 th through 12 th </li></ul>
  16. 16. PUEO Background <ul><li>Student/staff ratio varies by age group. 1:5 for younger children to 1:13 for oldest </li></ul><ul><li>Participants from both urban and rural backgrounds </li></ul><ul><li>Honolulu, Hawai’i at Punahou School </li></ul><ul><li>100+ hours of programming, depends upon level </li></ul>
  17. 17. PUEO Background <ul><li>Participants are all from low-income families. Some from immigrant households. </li></ul><ul><li>Total Budget: $411,906 </li></ul><ul><li>Supported through in-kind support from Punahou School and foundations </li></ul>
  18. 18. Key Evaluation Documents Logic Model & Evaluation Questions <ul><li>Why would a program use this document? For what purpose? </li></ul><ul><li>What would evaluators need to know to write this document? </li></ul><ul><li>Which program staff would be involved? When? How long? </li></ul>
  19. 19. Partnering with Evaluators Two Approaches <ul><li>Assess & Validate </li></ul><ul><ul><ul><li>Usually random assignment or quasi-experimental </li></ul></ul></ul><ul><li>Collaborative Stakeholder Approach </li></ul><ul><ul><ul><li>Program development focused </li></ul></ul></ul><ul><ul><ul><li>Process model </li></ul></ul></ul><ul><li>Not mutually exclusive </li></ul><ul><li>Be an informed consumer! </li></ul>
  20. 20. Contact Us Carl Ackerman [email_address] Beth Casey [email_address] Brenda McLaughlin [email_address]

×