Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Empowering youth to be evaluators: Involving Young People in Evaluating Informal Education Programs Presentation


Published on

  • Be the first to comment

  • Be the first to like this

Empowering youth to be evaluators: Involving Young People in Evaluating Informal Education Programs Presentation

  1. 1. 3/23/10 Empowering Youth To Be Evaluators: Involving young people in evaluating informal education programs Amy Grack Nelson, Evaluation & Research Associate Science Museum of Minnesota Overview   Overview of participatory evaluation   Participatory evaluation examples   Sampling of interactive techniques What is participatory evaluation? 1
  2. 2. 3/23/10 It’s All About Utility   Utility - one of the four essential features of all evaluations (Joint Committee on Standards for Educational Evaluation, 1994)   A way to help ensure use is to increase the primary intended users’ level of participation in the evaluation. (Cousins & Earl, 1995; Patton, 2008) Participatory Evaluation “Applied social research that involves trained evaluation personnel and practice-based decision makers working in partnership.” (Cousins & Earl, 1995, pg. 8) Core purpose  increasing use Characteristics of Participatory Evaluation   Balanced control of evaluation process   Involvement of primary users   Extensive participation throughout the evaluation (Cousins & Earl, 1995; Cousins & Whitmore, 1998) Interactive Evaluation Practice Continuum (King & Stevahn, 2002) 2
  3. 3. 3/23/10 Benefits of Participatory Evaluation Increases use of evaluation results by:   Enhancing relevance of the evaluation   Increasing understanding of the data   Increasing ownership of the findings (Cousins & Whitmore, 1998; King & Stevahn, 2002; Patton, 2008) Evaluation capacity building   Develop analytic and evaluative skills   Stakeholders develop a more “critical eye” (Cousins & Earl, 1992, 1995) Participatory Evaluation Examples Science Museum of Minnesota’s Kitty Andersen Youth Science Center Kitty Andersen Youth Science Center’s Park Crew Facilitate earth science and environmental education activities in the Big Back Yard and on outreaches 3
  4. 4. 3/23/10 Example 1: Summative Evaluation Youth will…   Learn about water related earth surface processes   Develop teaching skills   Learn about related science, technology, engineering, and math (STEM) careers Evaluation design   Observations and interviews of youth staff at the beginning and end of summer   Evaluation workshop to engage youth in results Reviewing the Work Keep/Change Discussion 4
  5. 5. 3/23/10 Incorporating Evaluation Data How often youth talked about why something is considered a pollutant (n=27) Keep/Change Discussion Benefits Limitations   Engages users with data to   Takes considerable amount think about successes and of time areas of improvement   Can be used to generate recommendations 5
  6. 6. 3/23/10 Youth Benefits Meaningful involvement in evaluation can help youth develop higher order thinking skills, specifically analytic and evaluative skills. (London et al., 2003)   Youth became more reflective of their work.   Youth comments reflected increased knowledge of the activities and confidence in sharing that knowledge with visitors.   Youth had a stronger sense of ownership and control. Adult Staff Benefits   Provided important feedback about the crew’s work.   Gained deeper understanding of the participants’ experience and could proactively identify and respond to their needs.   Increased understanding of evaluation and ability to interpret data and generate recommendations.   Developed capacity to include participatory evaluation in future work. 6
  7. 7. 3/23/10 Outcomes of the Process Participants experience a sense of empowerment and pride when they have an influence on the way programs are run and see their ideas acted upon. (Checkoway et al., 2003; Horsch et al., 2002; London et al., 2003)   Youth used suggestions to develop their own training.   They created a visitor survey and collected data.   Youth shared their ideas with a museum operations staff member. Example 2: Formative Evaluation of Outreaches Evaluation Process Identify daily Craft survey Pilot surveys objectives questions Discuss pilot data Enter and code Administer and revise data surveys surveys Analyze and Generate Improve outreach discuss data recommendations activities 7
  8. 8. 3/23/10 Organizational Requirements for PE   Evaluation must be valued   Sufficient time and resources   Commitment to organizational learning as a means to improvement   Motivated individuals   Interest and ability to learn evaluative skills (Cousins & Earl, 1992) Evaluator Requirements for PE   Sufficient technical and facilitator skills   Accessible for participatory activities and support   Necessary resources and time   Serve an instructional role   Motivation and commitment to participate   A tolerance for imperfection   Flexibility (Burke, 1998; Cousins & Earl, 1992; King, 1998) 8
  9. 9. 3/23/10 Interactive Techniques A sampling of interactive methods to gather and discuss evaluation data What youth do when they get stuck on a project (n=8) 9
  10. 10. 3/23/10 Interactive Graphs Benefits Limitations   Quick data collection   People may be   Everyone can see the influenced by others process and results   People may be hesitant   Can be used as a to place a rating where starting point for no one else has deeper conversations   Can see data by various characteristics (King, 2009) Carousel Sheets Benefits Limitations   Alternative to traditional   Participants may influence brainstorming each others’ responses   Lots of information in a   Tend to get first responses short timeframe and gut reaction; not   Quick way to see patterns deep and thoughtful   Promotes high involvement   Responses may be too brief   Involves users in analysis   May need to reanalyze some of the data (King, 2009) 10
  11. 11. 3/23/10 Questions? Amy Grack Nelson: To see a sampling of evaluation reports visit: 11