Inspire External Evaluation August18 08 09


Published on

Published in: Education, Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Inspire External Evaluation August18 08 09

  1. 1. External evaluation of the INSPIRE project : Innovative Science Pedagogy in Research and Education By Yves Beernaert & Magda Kirsch, Educonsult Comenius Transnational project INSPIRE Contract number: 2007-3430 / 001-001 Project number – 133867-LLP-1-2007-1-BE-COMENIUS-CMP
  2. 2. Why the INSPIRE project ? <ul><ul><li>It supports the Lisbon agenda 2000– 2010: </li></ul></ul><ul><ul><li>EU is to become the most competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth with more and better jobs and greater social cohesion </li></ul></ul><ul><li>It contributes to the benchmarks 2010 </li></ul><ul><ul><li>Halve the rate of early school leavers , in order to achieve an EU-average rate of 10%or less. </li></ul></ul><ul><ul><li>Halve the level of gender imbalance among graduates in MST & secure an significant increase of the total number of MST graduates, </li></ul></ul><ul><ul><li>Average % of 25-64 yrs in the EU with at least upper sec. education reaches 80%or more </li></ul></ul><ul><ul><li>Halve % of low-achieving 15 year olds in reading, mathematical and scientific literacy </li></ul></ul><ul><ul><li>Increase EU-average level of participation in LLL to at least 15% of working adult </li></ul></ul>
  3. 3. Lisbon agenda: Detailed workprogramme: 3 strategic objectives <ul><li>1. Improving the quality and effectiveness of education and training systems in Europe </li></ul><ul><li>2. Facilitating access of all to education and training systems </li></ul><ul><li>3. Opening Up education and training systems to the wider world </li></ul>
  4. 4. Strategic objective 1 : Improving the quality and effectiveness of education and training systems in Europe <ul><li>1.1. Improving education and training for teachers and trainers </li></ul><ul><li>1.2.Developing skills for the knowledge society </li></ul><ul><li>1.3. Ensuring access to ICT for everyone </li></ul><ul><li>1.4. Increasing the recruitment to scientific and technical studies </li></ul>
  5. 5. Strategic objective 2 : Facilitating access of all to education and training systems <ul><li>2.1. Open learning environments </li></ul><ul><li>2.2. Making learning more attractive </li></ul><ul><li>2.3. Supporting active citizenship, equal opportunities and social cohesion </li></ul>
  6. 6. Strategic objective 3 : Opening Up education and training systems to the wider world <ul><li>3.1. Strengthening the links with working life and research, and society at large </li></ul><ul><li>3.2. Developing the spirit of enterprise </li></ul><ul><li>3.3. Improving foreign language learning </li></ul><ul><li>3.4. Increasing mobility and exchange </li></ul><ul><li>3.5. Strengthening European co-operation </li></ul>
  7. 7. External evaluation of INSPIRE: AIMS <ul><li>To assess: </li></ul><ul><li>the quality & relevance of activities, processes implemented </li></ul><ul><li>the quality and relevance of the ‘educational ‘ or pedagogical’ outputs or the results </li></ul><ul><li>the quality and relevance of the dissemination / exploitation / valorisation actions </li></ul><ul><li>the sustainability of the project activities </li></ul><ul><li>the impact on various stakeholders </li></ul><ul><li>To identify key success and failure factors of the overall project </li></ul><ul><li>To propose measures that will improve effective project delivery in future partnerships </li></ul>
  8. 8. Elements of evaluation <ul><li>Relevance : contribution to specific needs at local, nat. level versus objectives of Comenius </li></ul><ul><li>Efficiency : </li></ul><ul><li>relationship input versus output </li></ul><ul><li>Effectiveness : </li></ul><ul><li>match between results and Comenius objectives </li></ul><ul><li>Impact / Sustainability </li></ul><ul><li>on teachers, decision-makers, education system </li></ul>
  9. 9. Two parts of the INSPIRE External evaluation <ul><li>a) Quality of the project, consortium and activities / outcomes, its organisation </li></ul><ul><li>b) Quality of the educational activities / outcomes and their (possible) impact on science education </li></ul>
  10. 10. External evaluation tools of b) pedagogical activities <ul><li>Protocol of experimentation: data collection </li></ul><ul><ul><li>Questionnaires for: </li></ul></ul><ul><ul><ul><ul><li>Teachers </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Pupils </li></ul></ul></ul></ul><ul><ul><ul><ul><li>National coordinators </li></ul></ul></ul></ul><ul><ul><ul><ul><li>School coordinators </li></ul></ul></ul></ul><ul><ul><li>Informal interviews with stakeholders </li></ul></ul><ul><li>Analysis of the data </li></ul><ul><li>Participation in meetings </li></ul><ul><li>Draft report plus feedback from stakeholders </li></ul>
  11. 11. Results of the external evaluation <ul><li>An external evaluation report </li></ul><ul><li>Parts of the report </li></ul><ul><ul><li>Description of aims, methodology </li></ul></ul><ul><ul><li>Evaluation of functioning, organisation of partnership </li></ul></ul><ul><ul><li>Evaluation of the pedagogical activities </li></ul></ul><ul><ul><ul><li>European Report </li></ul></ul></ul><ul><ul><ul><li>5 Country reports </li></ul></ul></ul>
  12. 12. Learning objects in MST teaching and learning Presentation of the European evaluation report of the pedagogical activities
  13. 13. Part 1:Participation in the project
  14. 14. Types of schools involved
  15. 15. Characteristics of the schools
  16. 16. Characteristics of the schools
  17. 17. Teachers participating in the survey
  18. 18. Experience of teachers in experiments
  19. 19. Pupils participating in the survey
  20. 20. Age of respondent pupils
  21. 21. Questions <ul><li>Still confusion about number of Austrian schools. How many exactly? </li></ul><ul><li>Nature of VET-schools: secondary or adult education? </li></ul><ul><li>Reasons for drop-out of German and Austrian schools? </li></ul><ul><ul><li>Please contact Magda or Yves during summer academy! </li></ul></ul>
  22. 22. Part 2: Organisation of project activities
  23. 23. Part 2: Organisation of project activities <ul><li>Overall thorough preparation /information </li></ul><ul><li>Coordinating school teachers informed about the project most refer to one (training) meeting or several meetings. </li></ul><ul><li>The national coordinators advised the teachers as to the LO to be used through e-mail, meetings, oral and written guidelines </li></ul><ul><li>Some NC indicated which LO could be used at the different levels of the schools. </li></ul>
  24. 24. Training of teachers and pupils
  25. 25. Training of teachers and pupils <ul><li>trainings of pupils lasted between 15 minutes and 10 hours </li></ul><ul><li>trainings for pupils given by a number of people involved such as ICT teachers, the school coordinator, the physics teacher etc. </li></ul><ul><li>training lasted between two hours and twenty hours. </li></ul><ul><li>trainings were given by national coordinator or the school coordinator. </li></ul>
  26. 26. Organisational facilitators
  27. 27. Evaluation tools teachers
  28. 28. Evaluation tools pupils
  29. 29. Coordination of evaluation activities: key =coordinator
  30. 30. Strong appreciation of School Coordinator
  31. 31. Strong appreciation of National Coordinator
  32. 32. Evaluation of monitoring
  33. 33. Any remarks ? <ul><li>Concerning preparation or information? </li></ul><ul><li>Concerning training? </li></ul><ul><li>Concerning monitoring and support? </li></ul><ul><ul><li>KEY role of national coordinators </li></ul></ul><ul><ul><li>KEY role of school coordinators </li></ul></ul>
  34. 34. Part III: Implementation <ul><li>Selection criteria of LO </li></ul><ul><li>Use of LO per category and per subject </li></ul><ul><li>Use of LO for extracurricular activities </li></ul>
  35. 35. Selection criteria of LO
  36. 36. Use of LO per subcategory
  37. 37. Most popular LO per subcategory <ul><li>Biology: </li></ul><ul><ul><li>B2-5-26   , “ Habitat of animals” used by 26 teachers, B3-5-27, the “ Virtual frog builder” ,  23 teachers </li></ul></ul><ul><li>Chemistry </li></ul><ul><ul><li>C3-5-15, “ Periodic table tetris” used 23 times and LO C4-5-16, “C haracteristics of materials , used 17 times </li></ul></ul>
  38. 38. Most popular LO per subcategory <ul><li>Mathematics </li></ul><ul><ul><li>LO M2-5-38, “ Maya numbers” used 23 times and LO M2-13-42 “ Trigonometry” used 21 times. </li></ul></ul><ul><li>Physics </li></ul><ul><ul><li>LO P4-5-4, “C onservation of energy” used 22 times LO P2-17-10, “F orces of the pendulum” 21 times. </li></ul></ul><ul><li>IT </li></ul><ul><ul><li>T3-13-55, “C omputer parts hangman” 33 times and LO T4-13-56 “C lick and learn computer parts” selected by 30 teachers </li></ul></ul>
  39. 39. Disciplines where LO were used
  40. 40. Extracurricular activities where LO were used
  41. 41. How did pupils work?
  42. 42. Average number of LO used per pupil
  43. 43. How were LO used? <ul><li>Average groups of around 20 pupils </li></ul><ul><li>Average time spent on LO : 1.5 – 2 hours </li></ul><ul><ul><li>Ranging from 15 minutes (languages) to 14 hours </li></ul></ul><ul><li>Mostly Inquiry-Based Science Education (IBSE) approach </li></ul><ul><li>88% of the teachers worked with special equipment </li></ul><ul><ul><li>Usually computer room (sometimes with Internet) </li></ul></ul><ul><ul><li>Sometimes beamer </li></ul></ul>
  44. 44. Questions <ul><li>How is it possible to use LO without special IT equipment? </li></ul><ul><li>What is meant by working for the LO 2 weeks, 3 months? Repeated use of same LO? </li></ul>
  45. 45. Part IV: Impact <ul><li>On teachers </li></ul><ul><ul><li>Expected impact on teaching before use of LO </li></ul></ul><ul><ul><li>Impact on teaching after use of LO </li></ul></ul><ul><ul><li>Impact on pupils’ competences </li></ul></ul><ul><ul><li>Impact on pupils’ attitudes </li></ul></ul><ul><li>On pupils </li></ul><ul><ul><li>Impact of MST before use of LO </li></ul></ul><ul><ul><li>Impact of MST after use of LO </li></ul></ul><ul><li>On schools </li></ul><ul><li>On education system </li></ul>
  46. 46. Teachers: Expected impact of LO
  47. 47. Teachers: Impact after use of LO
  48. 48. Teachers: Impact on teaching before and after
  49. 49. Teachers: Impact of LO on pupils’ appreciation of MST
  50. 50. Teachers: Impact on pupils’ attitudes
  51. 51. Pupils: Impact of MST before use of LO
  52. 52. Pupils’ appreciation of MST - gender
  53. 53. Pupils: Impact of MST after use of LO
  54. 54. Pupils: Impact of MST after use of LO – gender
  55. 55. Pupils: Impact of MST after use of LO – gender and age
  56. 56. Pupils: Impact of MST after use of LO – gender and age
  57. 57. Pupils: Impact of MST after use of LO youngest pupils– gender
  58. 58. Impact of LO: teachers vs. pupils
  59. 59. Impact of Inspire project on schools
  60. 60. Impact of Inspire project on education system
  61. 61. Impact of project: dissemination
  62. 62. Impact of project: dissemination
  63. 63. Impact of project: dissemination
  64. 64. Questions? <ul><li>What affected the impact most? </li></ul><ul><li>How could the impact be increased? </li></ul>
  65. 65. Part V: Success factors and facilitators
  66. 66. Differences Some results from the country chapters
  67. 67. Expectations
  68. 70. Conclusions <ul><li>The Inspire project has been a success thanks to: </li></ul><ul><ul><li>Motivation and hard work of all the teachers involved </li></ul></ul><ul><ul><li>Motivation and hard work of NC </li></ul></ul><ul><ul><li>Motivation and hard work of SC </li></ul></ul><ul><ul><li>Motivation of the pupils involved </li></ul></ul><ul><ul><li>The organisational structure (NC, SC etc.) set up by European Schoolnet </li></ul></ul><ul><ul><li>The involvement of several ministries (official status of project) </li></ul></ul>
  69. 71. Conclusions <ul><ul><li>The use of LO definitely enhances the quality of teaching and learning of MST </li></ul></ul><ul><ul><li>The use of LO enhances the acquisition of LLL skills </li></ul></ul><ul><ul><li>The impact of the LO is overall higher on boys than on girls (exception LT) </li></ul></ul><ul><ul><li>The younger the pupils are the higher the impact </li></ul></ul><ul><ul><li>The number of LO used does not affect the impact </li></ul></ul><ul><ul><li>The impact of the project is not only experienced on the schools involved but also on the education systems of the countries involved. </li></ul></ul>
  70. 72. Recommendations <ul><li>Use LO much more widely across the curriculum </li></ul><ul><ul><li>Selection of LO should not only be per subcategory but also per age group </li></ul></ul><ul><ul><li>Use LO as much as possible with younger age groups as they seem to have most impact with them </li></ul></ul><ul><li>Disseminate outcomes of Inspire project to the members of the MST Cluster of DG EAC </li></ul>
  71. 73. Thank you for your attention! Any more questions or remarks? Any supplementary information is welcome!