How to Increase the Quality of Learning Resources


Published on

Published in: Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

How to Increase the Quality of Learning Resources

  1. 1. NVU Conference 2009 How to increase the quality of learning resources S ystem for quality assurance iQTool Thorleif Hjeltnes,TISIP Geir Maribu, HiST
  2. 2. iQTool – Innovative eLearning Tool for Quality Training Material in VET LdV – multilateral project Development of Innovation Kick off meeting in Lisbon
  3. 3. Project Summary <ul><li>“ The aim of the project is to develop an open source software tool integrated in LMS(s), which is suitable to assess the teaching quality management of eLearning training programs and training materials for supporting the application of the quality measurement tool for institutions dealing with vocational training. </li></ul><ul><li>The project implementation integrates testing of software and pilot training of the training material. Thus the aims of the project are: </li></ul><ul><ul><li>to elaborate an eLearning quality tool which can be applied at European level and which enables the teaching of quality management, </li></ul></ul><ul><ul><li>to provide the management and quality assurance of vocational training with an effective tool. “ </li></ul></ul>
  4. 4. Partners <ul><li>SZÁMALK Education and Information Technology Ltd. - </li></ul><ul><li>The Research Foundation TISIP - </li></ul><ul><li>Technical University of Crete - </li></ul><ul><li>CENFIM -Vocational Training Centre for Metallurgy and Metalwork Industry (CENFIM) - </li></ul><ul><li>MTA SZTAKI, Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) - </li></ul><ul><li>F-BB Research Institute of Vocational Educational Training (f-bb) - </li></ul>
  5. 5. Work packages <ul><li>WP1: Management (SZAMALK, PL) </li></ul><ul><li>WP2: Research (TUC) </li></ul><ul><li>WP3: Software development (SZTAKI) </li></ul><ul><li>WP4: Educational content development (F-BB) </li></ul><ul><li>WP5: Testing and pilot training (CENFIM) </li></ul><ul><li>WP6: Quality and Evaluation (TISIP)‏ </li></ul><ul><li>WP7: Dissemination (SZAMALK) </li></ul>
  6. 6. WP2 <ul><li>Evaluation of Open Source e-learning platforms (LMS), </li></ul><ul><ul><ul><li>Aim: to choose an LMS to be used for trying out the iQTool system </li></ul></ul></ul><ul><li>Evaluation of Quality Management Systems, </li></ul><ul><ul><ul><li>Aim: to identify factors that influence quality assurance in e-learning and choose a quality management system to be used as model for the iQTool system </li></ul></ul></ul>
  7. 7. WP2 <ul><li>Evaluation of Open Source e-learning platforms (LMS)‏ </li></ul><ul><ul><li>Evaluation criteria and indicators </li></ul></ul><ul><ul><li>Selection of platforms to be evaluated </li></ul></ul><ul><ul><li>The evaluation </li></ul></ul><ul><ul><li>Propose suitable platform to be adopted by the project </li></ul></ul>
  8. 8. WP2 - Choosing an LMS: Criteria <ul><li>Active development Community </li></ul><ul><li>Existence of stable versions </li></ul><ul><li>Support English language </li></ul><ul><li>SCORM compliance </li></ul><ul><li>Support Learning objects as minimum granularity of learning content </li></ul><ul><li>Support for Core LMS functionality </li></ul><ul><li>Availability of information about their User Base </li></ul><ul><li>Existence of organization to support further development </li></ul><ul><li>Availability of third part reviews </li></ul><ul><li>Developers community: Number of posts, users, response time, quality of post and replies, friendliness of the community </li></ul><ul><li>Release activity: Number of releases per year, the significance of each release </li></ul><ul><li>Life span: Age of product, version number, if project is old has there been a cycle of redesign </li></ul><ul><li>Support, documentation, security, functionality and integration </li></ul>
  9. 9. WP2 - Choosing an LMS, continued <ul><li>Result: </li></ul><ul><li>The LMS system ILIAS was chosen </li></ul>
  10. 10. WP2 - Choosing a QA-system <ul><li>What to do: </li></ul><ul><ul><ul><li>Examine other EU projects for quality issues </li></ul></ul></ul><ul><ul><ul><li>Develop a questionnaire and make a survey among the target groups in the participating countries </li></ul></ul></ul><ul><li>What we want to find out: </li></ul><ul><ul><ul><li>Important factors that influence quality assurance in e-learning and vocational training </li></ul></ul></ul>
  11. 11. WP2 - The QA survey <ul><li>The questionnaire had 77 questions in seven parts: </li></ul><ul><ul><ul><li>Institutional support </li></ul></ul></ul><ul><ul><ul><li>Analysis </li></ul></ul></ul><ul><ul><ul><li>Design </li></ul></ul></ul><ul><ul><ul><li>Development </li></ul></ul></ul><ul><ul><ul><li>Delivery (teaching/learning activities)‏ </li></ul></ul></ul><ul><ul><ul><li>Student support </li></ul></ul></ul><ul><ul><ul><li>Evaluation, assessment </li></ul></ul></ul><ul><li>We received 114 answers </li></ul>
  12. 12. The answers excellent=1 good=2 understandable=3 poor=4 not at all=5 An additive index: summing up all answers given to those question into “one” variable: - Low value: high quality rating - High value: low quality rating
  13. 13. WP2 - Results from the QA survey <ul><li>Most questions have been answered more or less the same way, resulting in an almost equal distribution of answers </li></ul><ul><li>very few consider their procedures as “excellent” </li></ul><ul><li>most rate their procedures as “good” </li></ul><ul><li>second most answer to almost every question is “understandable” </li></ul><ul><li>This output indicates that there may be a demand in training institutions for a quality assurance tool as developed in the project IQTool . </li></ul>
  14. 14. WP 3 – Software development The aim of the work package is to develop a software tool for the evaluation of the training materials in LMS on the basis of the quality assurance methodology that was formed in WP2. The advantage of the integration of the evaluation system and LMS is that the quality related questionnaires can be directly assigned to the viewed learning object. Based on the answered questionnaires statistics can be made, which help to analyse the problems, and improve the quality of the training material.
  15. 15. Licences, development philosophy Every project result will be open, and freely available for commercial and non-commercial usage. Why? The widely used open source philosophy can help to spread and grow the technology. Licences. Every software component will be distributed under the terms of GPL, which is the mostly used open license. Every documentation and training material should be distributed under the terms of Creative Commons Attribution-Share Alike license.
  16. 16. Software architecture Repository User database Question database Answer database IQTool Application LMS iface Web interface Repository connector LMS LMS plugin
  17. 17. WP3 - Software architecture The 2 main part of the system is the IQTool system and the LMS . The IQTool system is divided 2 separated part: the repository and the IQTool application . The repository stores the questions, the answers and the user profiles of the IQTool users (LMS users are not stored here). The IQTool application contains the user interface of the administration system (add questions, manage users, etc.), and provides an interface for the LMS plugin.
  18. 18. WP4 – Educational content development <ul><li>Develop Course Material to teach the chosen QM system. </li></ul><ul><li>Module A: Theory about quality assurance </li></ul><ul><ul><ul><li>to present the general theory of Quality Management </li></ul></ul></ul><ul><ul><ul><li>to teach the basics of how to evaluate and assure the quality of online-based learning, including a pool of typical of questions </li></ul></ul></ul>
  19. 19. WP4 – Educational content development continued <ul><ul><li>Moduel B: Practical application about the theory </li></ul></ul><ul><ul><ul><li>to teach the application of the IQTool-Software and how to create questionnaires </li></ul></ul></ul><ul><ul><ul><li>the user gets familiar with the process, how to apply the iQTool software </li></ul></ul></ul><ul><ul><ul><li>the user will be able to measure the quality of eLearning courses and materials via Online-Questionnaires </li></ul></ul></ul>
  20. 20. Module A – some learning goals <ul><li>The user understands the difference between Quality Management and Quality Assurance. </li></ul><ul><li>The user knows about the main topics of each: QM and QA. </li></ul><ul><li>The user can follow the whole process of Quality Assurance. </li></ul><ul><li>The user comprehends the Bologna Process. </li></ul><ul><li>The user is able to apply QA in the field of eLearning. </li></ul><ul><li>The user can reproduce the different Quality Models, which are relevant for eLearning-applications. </li></ul><ul><li>The user will able to evaluate the quality of eLearning courses. </li></ul>
  21. 21. Module A <ul><li>Title: “Quality Assurance in eLearning” </li></ul><ul><li>Main topics: </li></ul><ul><ul><li>Overview of Quality Management and Quality Assurance </li></ul></ul><ul><ul><li>Methods and systems of Quality Management </li></ul></ul><ul><ul><li>Quality Management in VET and eLearning </li></ul></ul><ul><ul><li>Process of Quality Assurance </li></ul></ul><ul><ul><li>iQTool Quality model </li></ul></ul>VET-vocational Education and Training
  22. 22. Storyboard Module A <ul><li>71 screens with: </li></ul><ul><ul><ul><li>5 interactive exercises </li></ul></ul></ul><ul><ul><ul><li>10 charts </li></ul></ul></ul><ul><ul><ul><li>10 animations (approx. 5 - 8 in Flash)‏ </li></ul></ul></ul><ul><li>study time: 8 hours </li></ul>
  23. 23. Module B <ul><li>Title: “User Manual for the iQTool Software” </li></ul><ul><li>Main Topics: </li></ul><ul><ul><li>iQTool navigation </li></ul></ul><ul><ul><li>iQTool user roles </li></ul></ul><ul><ul><li>iQTool applications (administration, questions and questionnaires, statistics and publishing)‏ </li></ul></ul>
  24. 24. Storyboard Module B <ul><li>46 screens with: </li></ul><ul><ul><ul><li>14 interactive exercises, </li></ul></ul></ul><ul><ul><ul><li>18 screenshots </li></ul></ul></ul><ul><ul><ul><li>1 questions for self assessment </li></ul></ul></ul><ul><li>study time: 2 hours </li></ul>
  25. 25. Creating questionnaires
  26. 26. WP5 – Testing and Pilot training <ul><li>The two modules: </li></ul><ul><ul><ul><li>Module A: Quality Assurance in E-learning </li></ul></ul></ul><ul><ul><ul><li>Module B: User Manual for the iQTool software tool </li></ul></ul></ul><ul><li>e-Learning SCORM 1.2 compliant modules </li></ul><ul><ul><ul><li>Each Partner will install 8 e-Learning SCORM 1.2 compliant modules on ILIAS LMS Platform, in their local languages, to test the iQTool tool and the other modules (Module A and Module B), during the test phase </li></ul></ul></ul>
  27. 27. WP6 – some findings <ul><li>All partners works seriously and are committed to achieve the promised results to their best ability. </li></ul><ul><li>More focus on QA systems in general than on QA for e-learning resources and learning processes. </li></ul><ul><li>Difficult to glue together the different processes and results in each WP into a common product where one result build on top of the other. </li></ul><ul><li>Difficult to synchronize activities among the partners. </li></ul><ul><li>English as working language – misconceptions and different interpretations of the same text arises </li></ul>
  28. 28. Thank you <ul><li>More information of the project can be found at iQTool web site: </li></ul><ul><ul><ul><li> </li></ul></ul></ul>