Workshop 2  methods, video and interviews
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
460
On Slideshare
420
From Embeds
40
Number of Embeds
1

Actions

Shares
Downloads
1
Comments
0
Likes
0

Embeds 40

http://evaluationunawe.wikispaces.com 40

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Evaluation workshop 2 Aims for participants: 1.1 To understand evaluation context and key ideas 1.2 To share evaluation case studies 1.3 To learn about evaluation research and examples 2.1 To recap evaluation methods 2.2 To carry out (video) interviews 2.3 To consider data analysis 3.1 To review evaluation reports 3.2 To present demographic information using mapping software 3.3 To consider evaluation strategies 3.4 To present evaluation information
  • 2. Evaluation workshop 2 1 2
  • 3. Workshop 2 activities Workshop 2 Aims: 2.1 To recap evaluation methods 2.2 To carry out (video) interviews 2.3 To consider data analysis
  • 4. Day 2 Methods I have already used Never used Evaluation Methods notes Format: •Research examples •Methods to try out •Recommended reading
  • 5. Evaluation methods
  • 6. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK)
  • 7. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK)
  • 8. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Examples of front end evaluation 1. National Air and Space Museum front-end evaluation, Korn and Associates, 1992 http://informalscience.org/images/evaluation/report_32.pdf 2. Natural History Museum Meterorites gallery, 2005 http://www.nhm.ac.uk/resources-rx/files/meteorites-evaluation-11677.pdf 3. Havard-Smithsonian Center for Astrophysics: Front End Evaluation of EarthScope Panorama online game MacPherson and Associates, 2007 http://informalscience.org/images/evaluation/report_264.PDF
  • 9. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Learner Voice Hart’s ladder of engagement, 2008 Working with children importantly develops their view of democratic change, and they bring refreshing insight into your programme plans. Examples: • Sharing artwork to be shown in the Images of Nature gallery at the Natural History Museum. Activities in a local school explored children’s views about how best to display complex artworks in ways that would appeal to younger visitors. Outcome: child level questions prompted children to compare pictures with well known and straightforward icons, and to look out for motifs. • Sensation Science Centre in Dundee, Scotland, facilitated children to writing labels for the Discovery Boat exhibition.
  • 10. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK)
  • 11. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: consulting with pupils in advance of a programme
  • 12. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Research ethics: video permission
  • 13. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: Teacher panel session • Invite teachers through advertising on general websites e.g. TES connect, teacher unions • Share programme plans in a structured format • What’s in it for them? Combine with evening event
  • 14. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Teacher Panels Working closely with teachers at early stages in programming develops relationships and ensures that programme content meets a need and is aligned with curriculum goals. Example: Darwin Centre Teacher Panel, Natural History Museum, London Format: Teachers were invited to meetings before member events. Individuals within the organisation submitted activities they required input for, at the start of the academic year. These were scheduled according to project timescales, with three topics evaluated per panel. The Teacher Panel culminated in a celebration event at the opening of the Darwin Centre.
  • 15. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Recommended reading: Participatory Approaches Simon, N (2011) ‘The Participatory Museum’. Freely available online. http://www.participatorymuseum.org/ Innovation in front-end evaluation Exploring Social Media for front end evaluation, Jensen and Kelly 2009 Click image for hyperlink
  • 16. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Key questions: •What are your learning goals? •What are the requirements for reporting demographic data? •What is the scope for change? •Drop in events require formative evaluation to monitor audience segment attendance. •Pre booked events should have this data already collected. •When deciding how to ask for improvements, consider what is going to happen with the data. •Who will write it up? •Who needs to know? •What will they do with it? •Is there capacity for change? •Customer service- marketisation
  • 17. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Examples of formative evaluation 1.Science Café Online workshop evaluation Nanoscale Informal Science Education Le-Compte Hinley and Pattison, 2011 http://www.nisenet.org/sites/default/files/catalog/eval/uploads/2011/05/1353/sc 2. Investigating The Implementation of the Be A Scientist! Project in New York City and Los Angeles Formative Evaluation – Year One Center for Children and Technology Education Development Center, Inc., 2011 http://informalscience.org/images/evaluation/BAS_YR1_Eval_Report_Final.pdf 3. Data Visualizations: Year 1 Formative Evaluation Institute for learning innovation, 2011 http://informalscience.org/images/evaluation/Data_Viz.Sept_2011.Formative.fina
  • 18. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: Using google forms for an online survey 1 2 3 Benefits? Disadvantages?
  • 19. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: participant observation 1. Structure according to physical area of interest
  • 20. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: participant observation 2. Record teacher or pupil activity
  • 21. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: participant observation 3. Make a map of your space and tally behaviour http://www.familylearningforum.org/evaluation/types-of-evaluation/timing-tra
  • 22. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: participant observation 4. Record classroom activity
  • 23. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Recommended Reading http://psychology.ucdavis.edu/sommerb/sommerdemo/mapping/behmap.htm Links with marketing and psychology: http://www.mapping-museum-experience.com/en/about-project/project-descri “Psychogeography” http://www.hypothesisjournal.com/wp- content/uploads/2011/09/vol8no1-hj012.pdf Pearson, Barriault and Cochrane, 2010 Analysis of science communication engagement Innovation: Use of iPads in understanding children’s learning http://www2.2simple.com/apps? virtuemart_product_id=101&virtuemart_category_id=4
  • 24. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Work Package Objectives -> Evaluation -work package objectives were collaboratively developed into domains of learning for summative evaluation (March 2012) -This led to the development of the EU-UNAWE summative evaluation framework (August 2012) -National Project Co-ordinators improved the evaluation framework (October 2012)
  • 25. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) 1.1 The Program is addressed specially to disadvantaged children in different social environments. 1.2 The program intends to stimulate via images, playful activities, experiments, models and artistic activities the children´s awareness of astronomical objects and the place of the Earth as part of the Universe in a way of broadening the children´s view of the world. In doing so try to use throughout daily-life materials in the activities, experiments and models such that children can reproduce them at home. 1.3 Awaken the awareness of the uniqueness of the Earth and of the need for environmental preservation. 1.4 Motivate children to appreciate other cultures via stories and twinning activities with children living in other countries, making them aware of the place in which other children live, their common cultural features and differences. 1.5 Throughout stimulate children´s curiosity giving them the occasion and time to formulate their own questions related to the Universe. 1.6 Develop EUNAWE-contents to be linked to daily school curricula via training of the teachers.
  • 26. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) National Science Foundation Informal Science Education http://caise.insci.org/uploads/docs/Eval_Framework.pdf
  • 27. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Rationale for NSF Informal Science Education model 1. The variety in types of programs is expansive. Informal education and outreach activities can take place in schools, museums, the community, the media, and various other locations where people gather information and experience the world. Almost all ACC agencies had some type of program that was designed to generate awareness and engage the public in the agency’s work. Further, the types of activities varied considerably across programs. 2. The nature of these programs makes it difficult to conduct rigorous evaluation because, among other reasons: (1) the audience for these programs is diffuse and difficult to identify; (2) the multiple factors affecting and affected by these activities cannot be isolated for assessment; (3) the modest scale of these efforts does not warrant a costly assessment approach. There are examples of pre- and post- quasi-experimental evaluations of these programs, but it is extremely challenging to carry out rigorous studies to identify causality in these programs. 3. Third, despite all of these complexities many programs share the same or similar goals. Though the programs are varied, the Informal Education and Outreach group agreed on the importance of better interagency coordination and information sharing. Sharing best practices across agencies could offer significant benefits as these programs often bear a closer relationship to outreach and informal education programs in other agencies than they do to K–12 or postsecondary education programs in the same agency (U.S. Department of Education. [DoEd], 2007, p. 26.).
  • 28. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) EU projects have the same issues “Furthermore, due to budget and time constraints, the evaluation of initiatives is often limited. The interconnections between initiatives are very rare which effectively eliminates the possibilities for scaling-up and for the dissemination of the new ideas: the dynamics of “economy of scale” and the huge potential for real impact are simply not being exploited”. http://ec.europa.eu/research/science- society/document_library/pdf_06/report-rocard-on- science-education_en.pdf
  • 29. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) European model: framework for informal education
  • 30. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK)
  • 31. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK)
  • 32. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) March 2013: Evaluation guide published!
  • 33. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: interviewing pupils
  • 34. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: interviewing teachers
  • 35. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) Your task: to interview a colleague about the impact of participating in the UNAWE programme
  • 36. Day 2 Interviewing What has been the impact of being involved in EU UNAWE?
  • 37. Your questions Question Notes on answers Name
  • 38. Your questions Question Notes on answers Name Knowledge Skills Motivation Intercultural attitudes Legacy Guestnet Username: conf1310 Password: jooraeth
  • 39. 1.3 To learn about evaluation research and examples -Research about evaluation and examples (GK) To try: Using google forms for an online survey 1 2 3 Benefits? Disadvantages?
  • 40. What themes emerged?
  • 41. Day 2 Aims for participants: 1.1 To understand evaluation context and key ideas 1.2 To share evaluation case studies 1.3 To learn about evaluation research and examples 2.1 To recap evaluation methods 2.2 To carry out (video) interviews 2.3 To consider data analysis 3.1 To review evaluation reports 3.2 To present demographic information using mapping software 3.3 To consider evaluation strategies 3.4 To present evaluation information