Pathways Evaluation

Using Formative and Summative Evaluation to Improve Field Trip Programs
Kimberly M. Burtnyk
Amgen Center for Science Learning
California Science Center

Pre-/Post-visit classroom activities
Curriculum compatible
K8
CA standards
Gallery worksheets (atypical)
“ Chaperone Sheets”
Available free from our website
Think SCIENCE ! Pathways Components

Think SCIENCE ! Pathways Goals of Pathways Programs
Help teachers implement curriculum-compatible field trips
Increase educational guidance for field trips
Increase interactions with staff
Help chaperones facilitate
Increase concept understanding through chaperone facilitation

Formative Evaluation Spring 2002
Focus: Access, Use, Chaperone/student behavior
Summative Evaluation Timeline Spring 2003
Focus: Cognitive gains
Think SCIENCE ! Pathways Evaluation Plan

Recruiting
Groups with existing reservations
Questionnaires
Pathways usage issues
Field trip planning and behavior
Chaperone roles
Observations
Stopped/didn’t stop
Time spent--we want Pathways groups to stay longer
Panels Read, Discussions, Gestures
Evaluation Method

Recruiting
37 Schools recruited from March-June
Questionnaires
82 Returned (37 Teacher 45 Chaperone)
Observations
15 schools tagged for observation (8 PW, 7 Non-PW)
72 Chaperones observed
Formative Evaluation Results Summary

Only 3 of 8 PW recruits used it
Reasons
Technical difficulties
Materials looked too complicated and long
Confused about where to find them
Unaware of the materials
Timing
Formative Evaluation Questionnaires Hi-lites

Formative Evaluation Observations Hi-lites
72 Chaperones observed
Stopped/Did not Stop at Capsules

Formative Evaluation Observations Hi-lites Mean t: 66 s Mean t: 138 s Time spent at Capsules

Formative Evaluation Observations Hi-lites
Of the 35 chaperones who stopped...
Reading, Discussion, Gestures

Formative Evaluation Conclusions
Pathways usage was limited due to:
Access to computers/printers
Length and perceived complexity of the materials
Communication gaps between school administrators and teachers
BUT...
it did produce some significant behavioral
changes in chaperones who used it

How Did Formative Evaluation Impact the Program?
In-class/On-site activities separated
Shortened
Map removed
One activity removed (Gallery incomplete)
More direct link created
quick
easy
obvious

Summative Evaluation Method
Recruiting
Talked directly to lead teacher
Materials sent to groups
In-class visit prior to field trip
Pre/Post Test
Questionnaires --dropped
Observations
Gestures dropped

Summative Evaluation Results Summary
298 Students Recruited
217 had in-class visit
254 Pre- and Post-tests completed
28 PW
226 Non-PW
Observations
26 chaperones observed
ONLY 4 CHAPERONES USED PATHWAYS

Summative Evaluation Conclusions
Jury still out on cognitive impact of Pathways
The numbers are too small
Will resume attempts next year
Pre/Post test shows impact of In-class Visit
Micrometeoroid question Chi Square p<0.004 for those who had visit vs those who did not

1. Even with intensive intervention and support, teachers did not use PW
There were special circumstances in the A&S Gallery
And timing, curriculum, age groups were an issue too
BUT:
Are such materials worth spending $$ developing?
What do teachers really want/need, if anything? We don’t yet know.
2. Whether or not PW improves learning outcomes as the lit. suggests is still unknown
Warrants further testing and refinement of good and USEFUL field trip practices.
What did Formative and Summative Evaluation Teach us About Pathways?

3. We really need to learn more about teachers and chaperones
Can’t assume that teachers have access to net sources
Chaperones might be willing facilitators
4. We can apply statistics to tell us about the use and effectiveness of the program
Further refining statistical techniques
Learning curve
Exciting prospects
What did Formative and Summative Evaluation Teach us About Pathways?

http://joaomattar.com	55
http://blog.joaomattar.com	16

Pathways Evaluation

by KMBurtnyk on Feb 21, 2009

Accessibility

Upload Details

Usage Rights

2 Embeds 71

Statistics

Pathways Evaluation — Presentation Transcript