Top moments in UNAWE programmes                          Co                             un                                ...
   Framework for evaluation   Evaluation materials:    To do     1. Case study of your own programme- highlight       d...
Session structure1530 Introduction to evaluation frameworks1600 Sample materials1630 Workshop activity: gathering evidence...
“"the systematic application of social    research procedures for assessing the    conceptualization, design,    implement...
Front end      Formative   Summative                Process            Impact   Outcome
QuantitativeLarge sample sizes, closed questionsExample- Rosa’s likert scalesQualitativeSmall sample sizes, in depth, allo...
QuantitativeSurveysClosed questionsQualitative- smaller samplesCase studiesInterviewsObservationsVideo analysis
 Descriptive statistics Correlational analysis Statistical significanceIt is necessary for data to be comparable in ord...
A way to collect data that is consistent andsystematic across different types of programme
 Clarify Goals and Objectives- March 2012 Create a Model of Your Program Choose Methodology Choose Data Collection Met...
1.1 The Program is addressed specially to disadvantaged children in differentsocial environments.1.2 The program intends t...
 Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s...
Key questionsWhat are our assumptions abouta)What learning is?b)How participants might change as a result ofprogramme inte...
http://caise.insci.org/uploads/docs/Eval_Framework.pdf
1. The variety in types of programs is expansive. Informal education and outreach activitiescan take place in schools, mus...
“Furthermore, due to budget and timeconstraints,the evaluation of initiatives is oftenlimited. The interconnections betwee...
a   cy                    Leg              Awareness                                       of the                         ...
y  g ac          AwarenessLe                  of the                 Universe                (knowledge)                 E...
 Clarify Goals and Objectives Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) Anal...
Motivation
2. Scientific skills
 Mixed methods are required owing to variety  in programmes and ages Qualitative evidence needs to be gathered:-observat...
   Framework for evaluation   Evaluation materials:    To do     1. Case study of your own programme- highlight      di...
Using materials
Task 1: Submit a case study about your programmes anddevelopment: http://goo.gl/yJQY5These will form the basis for identif...
 Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s...
 Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s...
   Framework for evaluation   Evaluation materials:    To do     1. Case study of your own programme- highlight      di...
 Materials returned to GK by end of Dec 2012 Draft report by end of Feb 2013 Identify issues in data Modification and ...
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
EU-UNAWE Programme Evaluation
Upcoming SlideShare
Loading in...5
×

EU-UNAWE Programme Evaluation

410

Published on

Presentation

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
410
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

EU-UNAWE Programme Evaluation

  1. 1. Top moments in UNAWE programmes Co un tryCountryTeachers Ch ild re n How can we summarise and communicate impact in systematic ways? Grace Kimble and Dr Cecilia Scorza
  2. 2.  Framework for evaluation Evaluation materials: To do  1. Case study of your own programme- highlight disadvantage To email/ hand out and collect by end of Dec 2. Survey for teachers- give after training  3. Survey for pupils 8-10- give to teachers after training To visit and observe/listen  4. Activity for younger pupils: before/after  5. Observation sheet for early years (4-7)  Optional: long term impact interviews
  3. 3. Session structure1530 Introduction to evaluation frameworks1600 Sample materials1630 Workshop activity: gathering evidence/ timescales1700 Feedback- case studies1730 Finish
  4. 4. “"the systematic application of social research procedures for assessing the conceptualization, design, implementation, and utility of social intervention programs."http://ideas.stsci.edu/Evaluation.shtml
  5. 5. Front end Formative Summative Process Impact Outcome
  6. 6. QuantitativeLarge sample sizes, closed questionsExample- Rosa’s likert scalesQualitativeSmall sample sizes, in depth, allow for openended responsesMark and Libby’s questionnaires
  7. 7. QuantitativeSurveysClosed questionsQualitative- smaller samplesCase studiesInterviewsObservationsVideo analysis
  8. 8.  Descriptive statistics Correlational analysis Statistical significanceIt is necessary for data to be comparable in order for analysis
  9. 9. A way to collect data that is consistent andsystematic across different types of programme
  10. 10.  Clarify Goals and Objectives- March 2012 Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) Analyse Data Report and review Disseminate
  11. 11. 1.1 The Program is addressed specially to disadvantaged children in differentsocial environments.1.2 The program intends to stimulate via images, playful activities, experiments,models and artistic activities the children´s awareness of astronomical objectsand the place of the Earth as part of the Universe in a way of broadening the children´s view of the world. In doing so try to use throughout daily-life materials in theactivities, experiments and models such that children can reproduce them at home.1.3 Awaken the awareness of the uniqueness of the Earth and of the need forenvironmental preservation.1.4 Motivate children to appreciate other cultures via stories and twinningactivities with children living in other countries, making them aware of the place inwhich other children live, their common cultural features and differences.1.5 Throughout stimulate children´s curiosity giving them the occasion and time toformulate their own questions related to the Universe.1.6 Develop EUNAWE-contents to be linked to daily school curricula via trainingof the teachers.
  12. 12.  Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) Analyse Data Report and review Disseminate
  13. 13. Key questionsWhat are our assumptions abouta)What learning is?b)How participants might change as a result ofprogramme interventions?
  14. 14. http://caise.insci.org/uploads/docs/Eval_Framework.pdf
  15. 15. 1. The variety in types of programs is expansive. Informal education and outreach activitiescan take place in schools, museums, the community, the media, and various other locationswhere people gather information and experience the world. Almost all ACC agencies had sometype of program that was designed to generate awareness and engage the public in the agency’swork. Further, the types of activities varied considerably across programs.2. The nature of these programs makes it difficult to conduct rigorous evaluation because,among other reasons:(1) the audience for these programs is diffuse and difficult to identify;(2) the multiple factors affecting and affected by these activities cannot be isolated for assessment;(3) the modest scale of these efforts does not warrant a costly assessment approach.There are examples of pre- and post- quasi-experimental evaluations of these programs, but it isextremely challenging to carry out rigorous studies to identify causality in these programs.3. Third, despite all of these complexities many programs share the same or similar goals. Thoughthe programs are varied, the Informal Education and Outreach group agreed on the importanceof better interagency coordination and information sharing. Sharing best practices acrossagencies could offer significant benefits as these programs often bear a closer relationship tooutreach and informal education programs in other agencies than they do to K–12 orpostsecondary education programs in the same agency (U.S. Department of Education. [DoEd], 2007, p. 26.).
  16. 16. “Furthermore, due to budget and timeconstraints,the evaluation of initiatives is oftenlimited. The interconnections between initiativesare very rare which effectively eliminatesthe possibilities for scaling-up and for thedisseminationof the new ideas: the dynamics of“economy of scale” and the huge potential forreal impact are simply not being exploited”. http://ec.europa.eu/research/science-society/document_library/pdf_06/report-rocard-on-s
  17. 17. a cy Leg Awareness of the Universe (knowledge) EUNAWE fostering… Scientific Intercultural skills attitudes v at ion Moti“DOMAINS OF LEARNING”
  18. 18. y g ac AwarenessLe of the Universe (knowledge) EUNAWE fostering… Scientific Intercultural skills attitudes ion Motivat
  19. 19.  Clarify Goals and Objectives Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) Analyse Data Report and review Disseminate
  20. 20. Motivation
  21. 21. 2. Scientific skills
  22. 22.  Mixed methods are required owing to variety in programmes and ages Qualitative evidence needs to be gathered:-observation templates for children aged up to 7-before and after activities for children 6-10(Personal Meaning Mapping, John Falk)-drawings for ages 6-8, surveys ages 8-10
  23. 23.  Framework for evaluation Evaluation materials: To do  1. Case study of your own programme- highlight disadvantage To email/ hand out and collect by end of Dec  2. Survey for teachers- give after training  3. Survey for pupils 8-10- give to teachers after training To visit and observe/listen  4. Activity for younger pupils: before/after  5. Observation sheet for early years (4-7)  Optional: long term impact interviews
  24. 24. Using materials
  25. 25. Task 1: Submit a case study about your programmes anddevelopment: http://goo.gl/yJQY5These will form the basis for identifying examples for long term impactTask 2: Modify the pupil survey- translate. Check teachersurvey in paper and online: http://goo.gl/XejyPTask 3: Complete the checklist spreadsheet and returnExtra: add to observation framework
  26. 26.  Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) Analyse Data Report and review Disseminate
  27. 27.  Clarify Goals and Objectives- March Create a Model of Your Program Choose Methodology Choose Data Collection Method(s) –Dec 2012  identify opportunities for evaluation  check materials suitable for group  Seek support Analyse Data- GK: Jan- Feb 2013 Report and review- Mar 2013 Disseminate- Oct 2013
  28. 28.  Framework for evaluation Evaluation materials: To do  1. Case study of your own programme- highlight disadvantage To email/ hand out and collect by end of Dec  2. Survey for teachers- give after training  3. Survey for pupils 8-10- give to teachers after training To visit and observe/listen  4. Activity for younger pupils: before/after  5. Observation sheet for early years (4-7)  Optional: long term impact interviews
  29. 29.  Materials returned to GK by end of Dec 2012 Draft report by end of Feb 2013 Identify issues in data Modification and addressing gaps Mar 2013 Final report

×