Clearwater County SGFT Program EvaluationJillian Marotz MoroneyUniversity of IdahoBioregional Planning and Community Design
Roadmap  • Program History  • 2012 Program Evaluation   • Methods/Findings   • Recommendations   • Tools for Assessment of...
Program History                  Program History                  • Established 1961                  • Completely free to...
2012 Forestry Tour               • 32 students                • 15 boys/17 girls                • 11-12 years old         ...
Program Evaluation• Evening questionnaire asking  participants to rate program:    • Logistics    • Staff    • Facilities
Participant Assessmentof Program : Day 1                                               Wetlands    July 10, 2012          ...
Participant Assessmentof Program : Day 2    July 11, 2012       Event Assessment                                          ...
Participant Assessmentof Program : Day 3     64% of participants said that the       July 12, 2012 Event Assessment     lo...
Participant Assessmentof Program : Day 3    • Staff         • Adults         • Teens         • Guest SpeakersStaff Assessm...
Table 5: Facilities Assessment                                          Standard        Tour Event                 Mean   ...
Recommendations • Keep time of day in mind when planning the   schedule • Hands on activities with lectures • More trainin...
Recommendations Cont.• Communicate program themes to the presenters • Connect students to Idaho natural resources • Increa...
Knowledge AssessmentInstrument  • Knowledge gain and retention (20)    • Multiple Choice    • True/False  • Attitudes/Beha...
Pre-Test             Post-Test              Delayed Post-TestAdministered         Administered at the    Administered fall...
Knowledge Gain and Retention • 20 Questions • Questions submitted by presenters, program   organizer, state and federal cu...
Pre and Post-Test Averages  Pre-Test                             Post-Test  Control Average: 11.9                Control A...
Participants Pre, Post, and Delayed Post-test Scores             20             18             16             14Test Score...
Participant Averages by GenderPre-Test                      Post-Test           Delayed Post-TestMale Mean: 12.8          ...
Attitudes/Behaviors• 10 Statements• Statements from extensive literature review,  adapted to age group• Example:          ...
Change in Behaviors and Attitudes         5         4         3Rating         2         1         0             1   2    3...
5                                                                            Pre-TestControl v Participants Behaviors and ...
Forest Perceptions • “Draw yourself in a forest.” • Looked for before and after differences in   drawings  • Tree number/t...
“Draw yourself in a forest.”• 18/25 (72%) students who attended the tour  had more in depth drawings• 13/32 (41%) who took...
Camp Participants Drawings                             Pre-Test         Post-Test   Delayed Post-Test                     ...
Control Group Drawings                              Pre-Test   Delayed Post-Test                         Pre-Test        D...
Lessons and Suggestions• 2012 Program Evaluation • Methods   • Follow up questions to yes/no   • Communicate with presente...
Jillian Marotz Moroney     University of Idahomaro2762@vandals.uidaho.edu       Thank you!     Thesis Committee    Dr. Tam...
Upcoming SlideShare
Loading in...5
×

Clearwater County Sixth Grade Forestry Tour: Program Evaluation Methods and Tools

142
-1

Published on

The University of Idaho Extension has organized a Sixth Grade Forestry Tour in Clearwater County, Idaho for the last 51 years. During 2012, the program was evaluated to examine knowledge gained and retained by students in Clearwater County and track changes in behavior and attitudes about natural resources, and general satisfaction with the program’s logistics, staff, and facilities. All sixth grade students at Clearwater County schools were given a pretest in the spring consisting of questions about their knowledge of natural resources and their attitudes and behaviors towards natural resources. They were also asked to draw a picture of themselves in the forest. Students who attended the three-day tour were given the same test on the last day of the tour. A delayed posttest was given in the fall to all previously tested students, now beginning the seventh grade. Clearwater County students who did not attend the three-day tour served as the control group. The pre/posttest experimental design approach was used to determine the beginning knowledge of students, the knowledge gained as well as retained, and the attitude/behavioral changes which could be attributed to program attendance as opposed to maturity.
#nexc2012

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
142
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
1
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Clearwater County Sixth Grade Forestry Tour: Program Evaluation Methods and Tools

  1. 1. Clearwater County SGFT Program EvaluationJillian Marotz MoroneyUniversity of IdahoBioregional Planning and Community Design
  2. 2. Roadmap • Program History • 2012 Program Evaluation • Methods/Findings • Recommendations • Tools for Assessment of Participants • Knowledge • Behaviors/Attitudes • Perceptions • Lessons learned and suggestions
  3. 3. Program History Program History • Established 1961 • Completely free to participants • Three days, two nights • Teacher continuing education certification through University of Idaho • Covers topics including: • Forestry • Fish and wildlife • Range • Water Quality • Geology • Search and Rescue • Fire
  4. 4. 2012 Forestry Tour • 32 students • 15 boys/17 girls • 11-12 years old • Have parents who work in the logging industry: 9 • Born outside of Idaho: 19
  5. 5. Program Evaluation• Evening questionnaire asking participants to rate program: • Logistics • Staff • Facilities
  6. 6. Participant Assessmentof Program : Day 1 Wetlands July 10, 2012 Event Assessment Standard Tour Event Mean NezPerce Median Mode Fisheries DeviationPierce Pool • Heat/Sun 3.97 0.16 4 4Empire Lumber • Instructors 3.74 0.44 4 4 • ActivitiesOutdoor Skills 3.44 0.75 4 4 Tree IDResource Careers 3.28 0.76 3 3Dwarshak Fisheries 3.21 0.73 3 3Wetlands 3.18 0.88 3 4 RodeoNez Perce Fisheries 3.13 0.70 Grounds 3 3Tree ID 3.05 0.94 3 4Rodeo Grounds 3 0.83 3 4(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
  7. 7. Participant Assessmentof Program : Day 2 July 11, 2012 Event Assessment Standard Tour Event Mean Median Mode DeviationFire Control and Airplane Drop 3.95 0.22 4 4Mining Demo • 95% Said 3.92 they learned that 0.27 4 4Search and Rescue something this day 0.53 3.68 4 4ID Dept. of Lands 3.51 0.61 4 4Pierce Museum • Events with hands on 3.42 0.67 3 3Jaype Mill Pond activities scored highest 3.39 0.68 3.5 4Potlatch Forestry Management 3.39 0.69 3.5 4Forestry Consulting 3.32 0.70 3 4Group Activity 3.22 0.76 3 3NRCS 2.89 0.77 3 3(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
  8. 8. Participant Assessmentof Program : Day 3 64% of participants said that the July 12, 2012 Event Assessment lookout was their favorite part of Thursday. Tour Event Mean Standard Deviation Median Mode Bertha Hill Lookout 3.92 0.28 4 4 Outdoor Challenge 3.57 0.61 4 4 Natural Resource 3.51 0.61 4 4 Activity Group Activity 3.4 0.65 3 4 Woody Biomass 3.2 0.80 3 3 Logging Operations 3.06 1.01 3 4 Tree ID 2.94 1.01 3 4 (1=Poor, 2=Fair, 3=Good, 4=Excellent) The Tree ID quiz scored the lowest out ofN=32 any activity from the whole tour.http://idahoptv.org/outdoors/shows/eyesoftheforest/berthaHillLookout.cfm
  9. 9. Participant Assessmentof Program : Day 3 • Staff • Adults • Teens • Guest SpeakersStaff Assessment Standard Staff Member Mean Median Mode DeviationAdults 3.75 0.82 4 4Guest Speakers 3.74 0.61 4 4Teens 3.57 0.65 4 4(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
  10. 10. Table 5: Facilities Assessment Standard Tour Event Mean Median Mode DeviationBreakfast 3.97 0.17 4 4 OverallLunch 3.75 0.55 4 4 Favorite Meal: Pancakes andDinner 3.89 0.40 4 4 SpaghettiName Badges 3.41 0.82 4 4Tarps 3.31 0.75 3 3 LowestBathrooms 2.75 0.97 3 2 Ranking Facility:Transportation 3.47 0.70 4 4 BathroomsCampfire 3.72 0.45 4 4Schedule 3.28 0.70 4 4 Comments:Group Activities 3.41 0.78 4 4 “More Hiking!”Games 3.74 0.56 4 4 and “Less Hiking!”Skits 3.46 0.82 4 4(1=Poor, 2=Fair, 3=Good, 4=Excellent) N=32
  11. 11. Recommendations • Keep time of day in mind when planning the schedule • Hands on activities with lectures • More training/screening of teen leaders • More free time in the evenings
  12. 12. Recommendations Cont.• Communicate program themes to the presenters • Connect students to Idaho natural resources • Increase student interest in natural resources • Communicate what impact humans have on nature and natural resources • Increasing the comfort and connection students have with the outdoors
  13. 13. Knowledge AssessmentInstrument • Knowledge gain and retention (20) • Multiple Choice • True/False • Attitudes/Behaviors (10) • Likert Scale • Perceptions (2) • Drawing • Knowledge rating • Demographics (5) • Gender • Parental Occupation • Town/ County
  14. 14. Pre-Test Post-Test Delayed Post-TestAdministered Administered at the Administered fallspring 2012 to all end of the three day 2012 to all seventhsixth grade Forestry Tour to all grade students instudents in participants. Clearwater County,Clearwater County, Idaho.Idaho.
  15. 15. Knowledge Gain and Retention • 20 Questions • Questions submitted by presenters, program organizer, state and federal curriculum • Example: What is woody biomass used for? a. Generating electricity b. Heating homes and other buildings What are plants that c. Making liquid fuel annually lose their d. All of the above leaves called? a. Deciduous b. Nocturnal Fish have a very good sense c. Naked of smell. d. Coniferous a. True b. False
  16. 16. Pre and Post-Test Averages Pre-Test Post-Test Control Average: 11.9 Control Average: 11.8 Participant Average: 12.2 Participant Average: 13.8 Control N= 32 Participant N= 25
  17. 17. Participants Pre, Post, and Delayed Post-test Scores 20 18 16 14Test Score 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 Participants
  18. 18. Participant Averages by GenderPre-Test Post-Test Delayed Post-TestMale Mean: 12.8 Male Mean: 13.7 Male Mean: 14.1Female Mean: 11.7 Female Mean: 13.9 Female Mean: 13.5N Males= 12 N Females= 13
  19. 19. Attitudes/Behaviors• 10 Statements• Statements from extensive literature review, adapted to age group• Example: STRONGLY DON’T STRONGLY DISAGREE AGREE DISAGREE KNOW AGREE I am interested in forestry 1 2 3 4 5 and natural resources I talk about natural 1 2 3 4 5 resources with my friendsI feel good when I am outside 1 2 3 4 5 in nature I feel a strong connection to 1 2 3 4 5 the forests in Idaho
  20. 20. Change in Behaviors and Attitudes 5 4 3Rating 2 1 0 1 2 3 4 5 6 7 8 9 10 Question Number
  21. 21. 5 Pre-TestControl v Participants Behaviors and Attitudes 4 Rating 3 2 1 0 1 2 3 4 5 6 7 8 9 10 Question Number Post-Test 5 4 Rating 3 2 1 0 1 2 3 4 5 6 7 8 9 10 Question Number
  22. 22. Forest Perceptions • “Draw yourself in a forest.” • Looked for before and after differences in drawings • Tree number/type • Natural features
  23. 23. “Draw yourself in a forest.”• 18/25 (72%) students who attended the tour had more in depth drawings• 13/32 (41%) who took only pre and delayed post test showed more in depth drawings Pre-Test Post-Test Delayed Post-Test
  24. 24. Camp Participants Drawings Pre-Test Post-Test Delayed Post-Test Pre-Test Post-Test Delayed Post-Test
  25. 25. Control Group Drawings Pre-Test Delayed Post-Test Pre-Test Delayed Post-Test
  26. 26. Lessons and Suggestions• 2012 Program Evaluation • Methods • Follow up questions to yes/no • Communicate with presenters program goals and themes • Consider weather, age group, activities… • Tools for Assessment of Participants • Many tools (knowledge, behaviors, attitudes, perceptions, demographics….) • More tools means more data- pros and cons
  27. 27. Jillian Marotz Moroney University of Idahomaro2762@vandals.uidaho.edu Thank you! Thesis Committee Dr. Tamara Laninga Dr. Randy Brooks Dr. James Connors
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×