Research,	
  Evaluation,	
  
&	
  Visioning	
  Claire	
  Berezowitz,	
  University	
  of	
  Wisconsin-­‐Madison	
  
Andrea	
  Bontrager	
  Yoder,	
  University	
  of	
  Wisconsin-­‐Madison	
  
Beth	
  Hanna,	
  Community	
  GroundWorks	
  
Wisconsin	
  Farm	
  to	
  School	
  Summit	
  
Thursday,	
  January	
  29,	
  2015	
  
Wisconsin	
  Rapids,	
  WI	
  
Who	
  is	
  here	
  today?	
  
JEOPARDY	
  GAME!	
  
jeopardylabs.com/play/wisconsin-­‐farm-­‐to-­‐school	
  
FARM	
  TO	
  SCHOOL	
  ACTIVITY	
  TRACKER	
  
Project	
  History	
  
Why	
  track	
  program	
  activity?	
  
•  Common	
  language	
  
•  Common	
  quanOficaOon	
  
•  Compare	
  between	
  schools	
  
•  Track	
  development	
  within	
  	
  
	
  	
  	
  	
  school	
  across	
  Ome	
  
What	
  is	
  “Comprehensive”	
  F2S?	
  
•  Describe	
  tool:	
  
•  Four	
  domains:	
  
•  Variety	
  
•  Frequency	
  	
  
•  Minutes	
  	
  
Engagement	
  
acOviOes	
  
School	
  
gardens	
  
NutriOon,	
  	
  	
  	
  	
  
agriculture	
  
educaOon	
  
Local	
  foods	
  in	
  	
  
school	
  meals	
  
FARM	
  TO	
  SCHOOL	
  ACTIVITY	
  TRACKER	
  
Audience	
  parOcipaOon!	
  
Does	
  it	
  work?	
  
•  Monthly	
  acOvity	
  reports,	
  2010-­‐2011	
  
•  Enter	
  into	
  AcOvity	
  Tracker	
  
•  Program	
  managers	
  score	
  (1-­‐10)	
  each	
  site	
  
•  Correlate	
  domain	
  scores	
  with	
  expert	
  scores	
  
•  Three	
  months	
  of	
  acOvity	
  data,	
  Fall	
  2013	
  
•  5	
  raters	
  enter	
  into	
  AcOvity	
  Tracker	
  –	
  assess	
  for	
  inter-­‐rater	
  
reliability	
  according	
  to:	
  
•  Same	
  entries?	
  	
  
•  Same	
  domain-­‐level	
  scores?	
  
Domain	
  Scores	
  correlate	
  with	
  
Expert	
  Scores	
  
	
  Predictor	
   Spearman	
  Rank	
  
Correla=on	
  Coefficient	
  (r)	
  
Procurement:	
  Variety	
   	
  	
  	
  	
  	
  	
  	
  0.63***	
  
Procurement:	
  Frequency	
   	
  	
  	
  	
  	
  	
  	
  0.71***	
  
Classroom:	
  Number	
  of	
  lessons	
   	
  	
  	
  	
  	
  	
  	
  0.45**	
  
Classroom:	
  Number	
  of	
  minutes	
   	
  	
  	
  	
  	
  	
  	
  0.23	
  
Engagement:	
  Number	
  of	
  ac=vi=es	
   	
  	
  	
  	
  	
  	
  	
  0.71***	
  
Garden:	
  Number	
  of	
  visits	
   	
  	
  	
  	
  	
  	
  	
  0.42**	
  
Garden:	
  Number	
  of	
  minutes	
   data	
  not	
  collected	
  
***	
  p<.0001	
  	
  	
  	
  **	
  p<.001	
  	
  	
  	
  	
  *	
  p<.05	
  
BeneTits	
  of	
  tracking	
  F2S	
  
activity	
  
•  Common	
  language	
  
•  Common	
  quanOficaOon	
  
•  Compare	
  between	
  schools	
  
•  Track	
  development	
  within	
  	
  
	
  	
  	
  	
  school	
  across	
  Ome	
  
EVALUATION	
  FRAMEWORK	
  ACTIVITY	
  
Let’s	
  move	
  around!	
  
Wrap	
  up	
  
•  QuesOons/comments?	
  
•  How	
  can	
  we	
  move	
  F2S	
  evaluaOon	
  prioriOes	
  forward	
  in	
  
Wisconsin?	
  
•  Other	
  evaluaOon	
  tools:	
  
•  See	
  Wisconsin	
  F2S	
  Toolkit:	
  
	
  	
  	
  	
  hip://www.cias.wisc.edu/toolkits/	
  
Contact	
  information	
  
•  Claire	
  Berezowitz	
  
•  ckberezowitz@wisc.edu	
  
•  Andrea	
  Bontrager	
  Yoder	
  
•  ayoder@wisc.edu,	
  abontrageryoder@gmail.com	
  
•  Beth	
  Hanna	
  
•  beth@communitygroundworks.org	
  
Student	
  Outcomes	
  
0	
  
10	
  
20	
  
30	
  
40	
  
50	
  
0	
   1	
   ≥2	
  
%	
  of	
  Trays	
  
Percent	
  of	
  Trays	
  with	
  	
  
no	
  FV	
  disappearance	
  
0	
  
10	
  
20	
  
30	
  
40	
  
50	
  
0	
   1	
   ≥2	
  
%	
  of	
  Trays	
  
Percent	
  of	
  Trays	
  	
  
with	
  no	
  FV	
  items	
  
Fall	
  2010	
  
May	
  2011	
  
***	
  
***	
  
***	
  
***	
  
*	
  
0.0	
  
10.0	
  
20.0	
  
30.0	
  
40.0	
  
1	
   2	
   3	
   4	
   5	
   6	
   7	
   8	
   9	
   10	
   11	
   12	
   13	
   14	
  
Percent	
  of	
  trays	
  with	
  no	
  FV	
  items,	
  2010-­‐2011	
  
0.0	
  
10.0	
  
20.0	
  
30.0	
  
40.0	
  
1	
   2	
   3	
   4	
   5	
   6	
   7	
   8	
   9	
   10	
   11	
   12	
   13	
   14	
  
Percent	
  of	
  trays	
  with	
  no	
  FV	
  consumed,	
  2010-­‐2011	
  
Fall	
  2010	
  
Spring	
  2011	
  
Domain	
  Scores	
  correlate	
  with	
  
Student	
  Outcomes	
  
***	
  p<.0001	
  	
  	
  	
  **	
  p<.001	
  	
  	
  	
  	
  *	
  p<.05	
  
•  Correlates	
  with	
  improvements	
  in	
  Knowledge	
  scores:	
  
•  School	
  Meals:	
  Source,	
  Frequency	
  
•  Correlates	
  with	
  improvements	
  in	
  percent	
  of	
  students	
  with	
  no	
  
FV	
  consumed:	
  
•  School	
  Meals:	
  Source,	
  Variety	
  
•  Classroom	
  Educa<on:	
  Number	
  of	
  lessons	
  
•  Engagement	
  Ac<vi<es:	
  	
  Number	
  of	
  ac<vi<es	
  
•  Garden	
  Ac<vi<es:	
  Number	
  of	
  ac<vi<es	
  

F2 sat workshop f2s summit jan2015

  • 1.
    Research,  Evaluation,   &  Visioning  Claire  Berezowitz,  University  of  Wisconsin-­‐Madison   Andrea  Bontrager  Yoder,  University  of  Wisconsin-­‐Madison   Beth  Hanna,  Community  GroundWorks   Wisconsin  Farm  to  School  Summit   Thursday,  January  29,  2015   Wisconsin  Rapids,  WI  
  • 2.
    Who  is  here  today?  
  • 3.
  • 4.
    FARM  TO  SCHOOL  ACTIVITY  TRACKER  
  • 5.
  • 6.
    Why  track  program  activity?   •  Common  language   •  Common  quanOficaOon   •  Compare  between  schools   •  Track  development  within            school  across  Ome  
  • 7.
    What  is  “Comprehensive”  F2S?   •  Describe  tool:   •  Four  domains:   •  Variety   •  Frequency     •  Minutes     Engagement   acOviOes   School   gardens   NutriOon,           agriculture   educaOon   Local  foods  in     school  meals  
  • 8.
    FARM  TO  SCHOOL  ACTIVITY  TRACKER   Audience  parOcipaOon!  
  • 9.
    Does  it  work?   •  Monthly  acOvity  reports,  2010-­‐2011   •  Enter  into  AcOvity  Tracker   •  Program  managers  score  (1-­‐10)  each  site   •  Correlate  domain  scores  with  expert  scores   •  Three  months  of  acOvity  data,  Fall  2013   •  5  raters  enter  into  AcOvity  Tracker  –  assess  for  inter-­‐rater   reliability  according  to:   •  Same  entries?     •  Same  domain-­‐level  scores?  
  • 10.
    Domain  Scores  correlate  with   Expert  Scores    Predictor   Spearman  Rank   Correla=on  Coefficient  (r)   Procurement:  Variety                0.63***   Procurement:  Frequency                0.71***   Classroom:  Number  of  lessons                0.45**   Classroom:  Number  of  minutes                0.23   Engagement:  Number  of  ac=vi=es                0.71***   Garden:  Number  of  visits                0.42**   Garden:  Number  of  minutes   data  not  collected   ***  p<.0001        **  p<.001          *  p<.05  
  • 11.
    BeneTits  of  tracking  F2S   activity   •  Common  language   •  Common  quanOficaOon   •  Compare  between  schools   •  Track  development  within            school  across  Ome  
  • 12.
    EVALUATION  FRAMEWORK  ACTIVITY   Let’s  move  around!  
  • 13.
    Wrap  up   • QuesOons/comments?   •  How  can  we  move  F2S  evaluaOon  prioriOes  forward  in   Wisconsin?   •  Other  evaluaOon  tools:   •  See  Wisconsin  F2S  Toolkit:          hip://www.cias.wisc.edu/toolkits/  
  • 14.
    Contact  information   • Claire  Berezowitz   •  ckberezowitz@wisc.edu   •  Andrea  Bontrager  Yoder   •  ayoder@wisc.edu,  abontrageryoder@gmail.com   •  Beth  Hanna   •  beth@communitygroundworks.org  
  • 16.
    Student  Outcomes   0   10   20   30   40   50   0   1   ≥2   %  of  Trays   Percent  of  Trays  with     no  FV  disappearance   0   10   20   30   40   50   0   1   ≥2   %  of  Trays   Percent  of  Trays     with  no  FV  items   Fall  2010   May  2011   ***   ***   ***   ***   *  
  • 17.
    0.0   10.0   20.0   30.0   40.0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   Percent  of  trays  with  no  FV  items,  2010-­‐2011   0.0   10.0   20.0   30.0   40.0   1   2   3   4   5   6   7   8   9   10   11   12   13   14   Percent  of  trays  with  no  FV  consumed,  2010-­‐2011   Fall  2010   Spring  2011  
  • 18.
    Domain  Scores  correlate  with   Student  Outcomes   ***  p<.0001        **  p<.001          *  p<.05   •  Correlates  with  improvements  in  Knowledge  scores:   •  School  Meals:  Source,  Frequency   •  Correlates  with  improvements  in  percent  of  students  with  no   FV  consumed:   •  School  Meals:  Source,  Variety   •  Classroom  Educa<on:  Number  of  lessons   •  Engagement  Ac<vi<es:    Number  of  ac<vi<es   •  Garden  Ac<vi<es:  Number  of  ac<vi<es