Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.



Published on

Presentation given with Brian Merison (BILD) at EFODL International Conference on May 2007.

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this


  1. 1. European Self-Evaluation Tool for eLearning ‘ Evaluation for quality improvement’ EDUCATION AND CULTURE LEONARDO DA VINCI EFODL International Conference May 2007 Brian Merison – The British Institute for Learning & Development, UK Edwige Morin – VIDEOSCOP - Université Nancy 2, France
  2. 2. Project Promoter : EFODL Project partners : NADE (Norway) BEODL (Belgium) CESI Iberia (Spain) VMU (Lithuania) Le Préau (France) Vidéoscop/UN2 (France) ISQ (Portugal) B-I-L-D (UK) <ul><li>Aims of the project </li></ul><ul><li>Improve the quality of e-learning experiences </li></ul><ul><li>Development of multi-functional design tool for self-evaluation </li></ul><ul><li>Provide a guide for improving quality </li></ul><ul><li>Work with local Target Reference Groups </li></ul><ul><li>Dissemination at a European level </li></ul>3 products: Inventory Tool Manual E-learning course + effective evaluation = Better Quality learning
  3. 3. Better Quality learning? Results Learning experience
  4. 4. First challenge : S coping the project Who are the key stakeholders? Who will carry out the evaluation? ? Wider Public Third Parties (government agencies etc) External Facilitators External Providers Company Department Learning Department Company Management Learner
  5. 5. Second challenge: Using a quality model © EFQM Excellence Model
  6. 6. Level 1: Reactions – a ‘Smile Sheet’ completed by the learner.  Level 2: Learning outcomes – attempts to measure what has been learned (skills, knowledge and/or attitudes) Level 3: Transfer of learning to the everyday environment – changes in behaviour Level 4: Results for the organisation such as increased productivity, increased sales, lower costs etc. Third challenge: Using an evaluation model The Kirkpatrick model
  7. 7. Constructing the framework A range of stakeholders ? – an unanswered question – who will carry out the evaluation ? EFQM Excellence Model Kirkpatrick evaluation Model The Learner
  8. 8. Constructing the framework A range of stakeholder interests The Learner Questionnaire Designer Learning resources Learning processes Results
  9. 9. Learning Resources Learning Processes Learning Results Main Criteria Process 1 Process 2 Process 3 Criteria A questionnaire based on a three level structure Process 4 Sub-Process a Sub-Process b Sub-Process d Sub-Process e Sub-Process c Sub - Criteria Statements Questions / Statements
  10. 10. Learning RESOURCES Part I 5 Criteria 3 2 3 3 2 79 Items of evaluation Information Electronic learning environment Physical environment Face to Face environment Learning materials 19 23 12 12 13
  11. 11. Learning PROCESSES Organisation of services Knowledge evaluation Part II 4 Criteria 4 3 17 12 85 Items of evaluation Learning activities 7 42 Learner Support 3 14
  12. 12. Learning RESULTS Part III 3 Criteria Learner’s competencies 1 4 45 Items of evaluation Learner’s knowledge 2 7 Ability to learn 5 34
  13. 13. Question Applicable/Not applicable Rating Priority of improvement Related to main crtieria / criteria/ subcriteria How important is this topic for you in this course ? What is your evaluation of this topic in your course ? Is improvement necessary ? 1.1 Accessibility 1.1.1 Access to relevant information on learning opportunities 1 2 3 4     1.1.2 Means to personalise the information given to meet my needs and profile 1 2 3 4 1 2 3 4   1.2 Description of the learning supply 1.2.1 Availability of up to date information on the course 1 2 3 4 1 2 3 4 Yes No
  14. 14. Tool functionalities The stakeholders The learners SEVAQ tool administrator SEVAQ tool Enables the designer with instalment and setting the tool The entity who designs the questionnaire to be applied to the learner The final recipients of the results. The people for whom the evaluation is really set-up. Three functions in the tool 1 – designing the questionnaire 2 – producing 3 – consulting results 1 – Securing the purpose of the evaluation 2 – Building the questionnaire framework to be applied 3 – Applying the questionnaires 6 – Building an action plan for improvement 5 – Analysing the results 4 – Getting the results The questionnaire Designer
  15. 15. Home page Answers to a questionnaire Create and manage questionnaires Manage the tool
  16. 16. The designer using the sevaq tool Create and manage questionnaires Create and manage learners Quick link menu
  17. 17. Create a new questionnaire <ul><li>Three steps to create your questionnaire </li></ul>Step 1 Setup your questionnaire Step 2 Define the objectives of your questionnaire Step 3 Develop your questionnaire
  18. 18. <ul><li>Assign a title, - give dates of beginning and end - Choose a language - Define the type of questionning (short or long) </li></ul>Create a new questionnaire > Step 1
  19. 19. <ul><li>select the Main criteria </li></ul><ul><li>and criteria for your questionnaire </li></ul>Create a new questionnaire > Step 2
  20. 20. - select the subcriteria - select the questions - write new questions - validate the questionnaire Create a new questionnaire > Step 3
  21. 21. Manage your questionnaires <ul><li>modify your questionnaire - save your questionnaire as - preview your questionnaire - archive your questionnaire - delete your questionnaire </li></ul>
  22. 22. Three choices 1 – completely anonymous (one login and pwd) 2 – anonymous amongst a targeted public belonging to the same course (one login and pwd for one course) 3 - identified for each learner (one login and pwd/ learner) Learners & groups
  23. 23. - choose your users or groups - publish your questionnaire - receive an invitation email Link questionnaire to learners or groups
  24. 24. - choose the questionnaire to answer - answer the questions - add your comments - fill up the profilling - validate your answers Answer a questionnaire
  25. 25. Questionnaires results <ul><li>raw figures - analysis diagrams - profiling results - learner’s comments </li></ul>
  26. 26. <ul><li>Summary: </li></ul><ul><li>Combines EFQM and Kirkpatrick for an innovative framework and approach </li></ul><ul><li>Identifies the various stakeholders with an interest in evaluation </li></ul><ul><li>Provides a tool for the designer prompting coverage of all relevant issues </li></ul><ul><li>Allows flexibility for the designer to add and customise questions to give relevance to all applications. </li></ul>EFODL International Conference May 2007
  27. 27. EFODL Project manager Helga Van Heysbroeck hvheysbr @ vdab .be www. efodl .net Brian Merison brian @ british Edwige Morin Edwige .Morin@ univ -nancy2. fr For additional questions, a personal test log-in or further dialogue: WWW.SEVAQ.COM Thank you for your attention!