How to Increase the Quality of Learning ResourcesPresentation Transcript
NVU Conference 2009 How to increase the quality of learning resources S ystem for quality assurance iQTool Thorleif Hjeltnes,TISIP Geir Maribu, HiST
iQTool – Innovative eLearning Tool for Quality Training Material in VET LdV – multilateral project Development of Innovation Kick off meeting in Lisbon
“ The aim of the project is to develop an open source software tool integrated in LMS(s), which is suitable to assess the teaching quality management of eLearning training programs and training materials for supporting the application of the quality measurement tool for institutions dealing with vocational training.
The project implementation integrates testing of software and pilot training of the training material. Thus the aims of the project are:
to elaborate an eLearning quality tool which can be applied at European level and which enables the teaching of quality management,
to provide the management and quality assurance of vocational training with an effective tool. “
SZÁMALK Education and Information Technology Ltd. - http://www.szamalk.hu
The Research Foundation TISIP - http://www.tisip.no
Technical University of Crete - http://www.music.tuc.gr
CENFIM -Vocational Training Centre for Metallurgy and Metalwork Industry (CENFIM) - http://www.cenfim.pt
MTA SZTAKI, Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) - http://www.sztaki.hu/elearning
F-BB Research Institute of Vocational Educational Training (f-bb) - http://www.f-bb.de
WP1: Management (SZAMALK, PL)
WP2: Research (TUC)
WP3: Software development (SZTAKI)
WP4: Educational content development (F-BB)
WP5: Testing and pilot training (CENFIM)
WP6: Quality and Evaluation (TISIP)
WP7: Dissemination (SZAMALK)
Evaluation of Open Source e-learning platforms (LMS),
Aim: to choose an LMS to be used for trying out the iQTool system
Evaluation of Quality Management Systems,
Aim: to identify factors that influence quality assurance in e-learning and choose a quality management system to be used as model for the iQTool system
Evaluation of Open Source e-learning platforms (LMS)
Evaluation criteria and indicators
Selection of platforms to be evaluated
Propose suitable platform to be adopted by the project
WP2 - Choosing an LMS: Criteria
Active development Community
Existence of stable versions
Support English language
Support Learning objects as minimum granularity of learning content
Support for Core LMS functionality
Availability of information about their User Base
Existence of organization to support further development
Availability of third part reviews
Developers community: Number of posts, users, response time, quality of post and replies, friendliness of the community
Release activity: Number of releases per year, the significance of each release
Life span: Age of product, version number, if project is old has there been a cycle of redesign
Support, documentation, security, functionality and integration
WP2 - Choosing an LMS, continued
The LMS system ILIAS was chosen
WP2 - Choosing a QA-system
What to do:
Examine other EU projects for quality issues
Develop a questionnaire and make a survey among the target groups in the participating countries
What we want to find out:
Important factors that influence quality assurance in e-learning and vocational training
WP2 - The QA survey
The questionnaire had 77 questions in seven parts:
Delivery (teaching/learning activities)
We received 114 answers
The answers excellent=1 good=2 understandable=3 poor=4 not at all=5 An additive index: summing up all answers given to those question into “one” variable: - Low value: high quality rating - High value: low quality rating
WP2 - Results from the QA survey
Most questions have been answered more or less the same way, resulting in an almost equal distribution of answers
very few consider their procedures as “excellent”
most rate their procedures as “good”
second most answer to almost every question is “understandable”
This output indicates that there may be a demand in training institutions for a quality assurance tool as developed in the project IQTool .
WP 3 – Software development The aim of the work package is to develop a software tool for the evaluation of the training materials in LMS on the basis of the quality assurance methodology that was formed in WP2. The advantage of the integration of the evaluation system and LMS is that the quality related questionnaires can be directly assigned to the viewed learning object. Based on the answered questionnaires statistics can be made, which help to analyse the problems, and improve the quality of the training material.
Licences, development philosophy Every project result will be open, and freely available for commercial and non-commercial usage. Why? The widely used open source philosophy can help to spread and grow the technology. Licences. Every software component will be distributed under the terms of GPL, which is the mostly used open license. Every documentation and training material should be distributed under the terms of Creative Commons Attribution-Share Alike license.
Software architecture Repository User database Question database Answer database IQTool Application LMS iface Web interface Repository connector LMS LMS plugin
WP3 - Software architecture The 2 main part of the system is the IQTool system and the LMS . The IQTool system is divided 2 separated part: the repository and the IQTool application . The repository stores the questions, the answers and the user profiles of the IQTool users (LMS users are not stored here). The IQTool application contains the user interface of the administration system (add questions, manage users, etc.), and provides an interface for the LMS plugin.
WP4 – Educational content development
Develop Course Material to teach the chosen QM system.
Module A: Theory about quality assurance
to present the general theory of Quality Management
to teach the basics of how to evaluate and assure the quality of online-based learning, including a pool of typical of questions
WP4 – Educational content development continued
Moduel B: Practical application about the theory
to teach the application of the IQTool-Software and how to create questionnaires
the user gets familiar with the process, how to apply the iQTool software
the user will be able to measure the quality of eLearning courses and materials via Online-Questionnaires
Module A – some learning goals
The user understands the difference between Quality Management and Quality Assurance.
The user knows about the main topics of each: QM and QA.
The user can follow the whole process of Quality Assurance.
The user comprehends the Bologna Process.
The user is able to apply QA in the field of eLearning.
The user can reproduce the different Quality Models, which are relevant for eLearning-applications.
The user will able to evaluate the quality of eLearning courses.
Title: “Quality Assurance in eLearning”
Overview of Quality Management and Quality Assurance
Methods and systems of Quality Management
Quality Management in VET and eLearning
Process of Quality Assurance
iQTool Quality model
VET-vocational Education and Training
Storyboard Module A
71 screens with:
5 interactive exercises
10 animations (approx. 5 - 8 in Flash)
study time: 8 hours
Title: “User Manual for the iQTool Software”
iQTool user roles
iQTool applications (administration, questions and questionnaires, statistics and publishing)
Storyboard Module B
46 screens with:
14 interactive exercises,
1 questions for self assessment
study time: 2 hours
WP5 – Testing and Pilot training
The two modules:
Module A: Quality Assurance in E-learning
Module B: User Manual for the iQTool software tool
e-Learning SCORM 1.2 compliant modules
Each Partner will install 8 e-Learning SCORM 1.2 compliant modules on ILIAS LMS Platform, in their local languages, to test the iQTool tool and the other modules (Module A and Module B), during the test phase
WP6 – some findings
All partners works seriously and are committed to achieve the promised results to their best ability.
More focus on QA systems in general than on QA for e-learning resources and learning processes.
Difficult to glue together the different processes and results in each WP into a common product where one result build on top of the other.
Difficult to synchronize activities among the partners.
English as working language – misconceptions and different interpretations of the same text arises
More information of the project can be found at iQTool web site: