NVU Conference 2009 How to increase the quality of learning resources S ystem for quality assurance iQTool Thorleif Hjeltnes,TISIP Geir Maribu, HiST

iQTool – Innovative eLearning Tool for Quality Training Material in VET LdV – multilateral project Development of Innovation Kick off meeting in Lisbon

Project Summary “ The aim of the project is to develop an open source software tool integrated in LMS(s), which is suitable to assess the teaching quality management of eLearning training programs and training materials for supporting the application of the quality measurement tool for institutions dealing with vocational training. The project implementation integrates testing of software and pilot training of the training material. Thus the aims of the project are: to elaborate an eLearning quality tool which can be applied at European level and which enables the teaching of quality management, to provide the management and quality assurance of vocational training with an effective tool. “

“ The aim of the project is to develop an open source software tool integrated in LMS(s), which is suitable to assess the teaching quality management of eLearning training programs and training materials for supporting the application of the quality measurement tool for institutions dealing with vocational training.

The project implementation integrates testing of software and pilot training of the training material. Thus the aims of the project are:

to elaborate an eLearning quality tool which can be applied at European level and which enables the teaching of quality management,

to provide the management and quality assurance of vocational training with an effective tool. “

Partners SZÁMALK Education and Information Technology Ltd. - http://www.szamalk.hu The Research Foundation TISIP - http://www.tisip.no Technical University of Crete - http://www.music.tuc.gr CENFIM -Vocational Training Centre for Metallurgy and Metalwork Industry (CENFIM) - http://www.cenfim.pt MTA SZTAKI, Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) - http://www.sztaki.hu/elearning F-BB Research Institute of Vocational Educational Training (f-bb) - http://www.f-bb.de

SZÁMALK Education and Information Technology Ltd. - http://www.szamalk.hu

The Research Foundation TISIP - http://www.tisip.no

Technical University of Crete - http://www.music.tuc.gr

CENFIM -Vocational Training Centre for Metallurgy and Metalwork Industry (CENFIM) - http://www.cenfim.pt

MTA SZTAKI, Computer and Automation Research Institute, Hungarian Academy of Sciences (MTA SZTAKI) - http://www.sztaki.hu/elearning

F-BB Research Institute of Vocational Educational Training (f-bb) - http://www.f-bb.de

Work packages WP1: Management (SZAMALK, PL) WP2: Research (TUC) WP3: Software development (SZTAKI) WP4: Educational content development (F-BB) WP5: Testing and pilot training (CENFIM) WP6: Quality and Evaluation (TISIP)‏ WP7: Dissemination (SZAMALK)

WP1: Management (SZAMALK, PL)

WP2: Research (TUC)

WP3: Software development (SZTAKI)

WP4: Educational content development (F-BB)

WP5: Testing and pilot training (CENFIM)

WP6: Quality and Evaluation (TISIP)‏

WP7: Dissemination (SZAMALK)

WP2 Evaluation of Open Source e-learning platforms (LMS), Aim: to choose an LMS to be used for trying out the iQTool system Evaluation of Quality Management Systems, Aim: to identify factors that influence quality assurance in e-learning and choose a quality management system to be used as model for the iQTool system

Evaluation of Open Source e-learning platforms (LMS),

Aim: to choose an LMS to be used for trying out the iQTool system

Evaluation of Quality Management Systems,

Aim: to identify factors that influence quality assurance in e-learning and choose a quality management system to be used as model for the iQTool system

WP2 Evaluation of Open Source e-learning platforms (LMS)‏ Evaluation criteria and indicators Selection of platforms to be evaluated The evaluation Propose suitable platform to be adopted by the project

Evaluation of Open Source e-learning platforms (LMS)‏

Evaluation criteria and indicators

Selection of platforms to be evaluated

The evaluation

Propose suitable platform to be adopted by the project

WP2 - Choosing an LMS: Criteria Active development Community Existence of stable versions Support English language SCORM compliance Support Learning objects as minimum granularity of learning content Support for Core LMS functionality Availability of information about their User Base Existence of organization to support further development Availability of third part reviews Developers community: Number of posts, users, response time, quality of post and replies, friendliness of the community Release activity: Number of releases per year, the significance of each release Life span: Age of product, version number, if project is old has there been a cycle of redesign Support, documentation, security, functionality and integration

Active development Community

Existence of stable versions

Support English language

SCORM compliance

Support Learning objects as minimum granularity of learning content

Support for Core LMS functionality

Availability of information about their User Base

Existence of organization to support further development

Availability of third part reviews

Developers community: Number of posts, users, response time, quality of post and replies, friendliness of the community

Release activity: Number of releases per year, the significance of each release

Life span: Age of product, version number, if project is old has there been a cycle of redesign

Support, documentation, security, functionality and integration

WP2 - Choosing an LMS, continued Result: The LMS system ILIAS was chosen

Result:

The LMS system ILIAS was chosen

WP2 - Choosing a QA-system What to do: Examine other EU projects for quality issues Develop a questionnaire and make a survey among the target groups in the participating countries What we want to find out: Important factors that influence quality assurance in e-learning and vocational training

What to do:

Examine other EU projects for quality issues

Develop a questionnaire and make a survey among the target groups in the participating countries

What we want to find out:

Important factors that influence quality assurance in e-learning and vocational training

WP2 - The QA survey The questionnaire had 77 questions in seven parts: Institutional support Analysis Design Development Delivery (teaching/learning activities)‏ Student support Evaluation, assessment We received 114 answers

The questionnaire had 77 questions in seven parts:

Institutional support

Analysis

Design

Development

Delivery (teaching/learning activities)‏

Student support

Evaluation, assessment

We received 114 answers

The answers excellent=1 good=2 understandable=3 poor=4 not at all=5 An additive index: summing up all answers given to those question into “one” variable: - Low value: high quality rating - High value: low quality rating

WP2 - Results from the QA survey Most questions have been answered more or less the same way, resulting in an almost equal distribution of answers very few consider their procedures as “excellent” most rate their procedures as “good” second most answer to almost every question is “understandable” This output indicates that there may be a demand in training institutions for a quality assurance tool as developed in the project IQTool .

Most questions have been answered more or less the same way, resulting in an almost equal distribution of answers

very few consider their procedures as “excellent”

most rate their procedures as “good”

second most answer to almost every question is “understandable”

This output indicates that there may be a demand in training institutions for a quality assurance tool as developed in the project IQTool .

WP 3 – Software development The aim of the work package is to develop a software tool for the evaluation of the training materials in LMS on the basis of the quality assurance methodology that was formed in WP2. The advantage of the integration of the evaluation system and LMS is that the quality related questionnaires can be directly assigned to the viewed learning object. Based on the answered questionnaires statistics can be made, which help to analyse the problems, and improve the quality of the training material.

Licences, development philosophy Every project result will be open, and freely available for commercial and non-commercial usage. Why? The widely used open source philosophy can help to spread and grow the technology. Licences. Every software component will be distributed under the terms of GPL, which is the mostly used open license. Every documentation and training material should be distributed under the terms of Creative Commons Attribution-Share Alike license.

Software architecture Repository User database Question database Answer database IQTool Application LMS iface Web interface Repository connector LMS LMS plugin

WP3 - Software architecture The 2 main part of the system is the IQTool system and the LMS . The IQTool system is divided 2 separated part: the repository and the IQTool application . The repository stores the questions, the answers and the user profiles of the IQTool users (LMS users are not stored here). The IQTool application contains the user interface of the administration system (add questions, manage users, etc.), and provides an interface for the LMS plugin.

WP4 – Educational content development Develop Course Material to teach the chosen QM system. Module A: Theory about quality assurance to present the general theory of Quality Management to teach the basics of how to evaluate and assure the quality of online-based learning, including a pool of typical of questions

Develop Course Material to teach the chosen QM system.

Module A: Theory about quality assurance

to present the general theory of Quality Management

to teach the basics of how to evaluate and assure the quality of online-based learning, including a pool of typical of questions

WP4 – Educational content development continued Moduel B: Practical application about the theory to teach the application of the IQTool-Software and how to create questionnaires the user gets familiar with the process, how to apply the iQTool software the user will be able to measure the quality of eLearning courses and materials via Online-Questionnaires

Moduel B: Practical application about the theory

to teach the application of the IQTool-Software and how to create questionnaires

the user gets familiar with the process, how to apply the iQTool software

the user will be able to measure the quality of eLearning courses and materials via Online-Questionnaires

Module A – some learning goals The user understands the difference between Quality Management and Quality Assurance. The user knows about the main topics of each: QM and QA. The user can follow the whole process of Quality Assurance. The user comprehends the Bologna Process. The user is able to apply QA in the field of eLearning. The user can reproduce the different Quality Models, which are relevant for eLearning-applications. The user will able to evaluate the quality of eLearning courses.

The user understands the difference between Quality Management and Quality Assurance.

The user knows about the main topics of each: QM and QA.

The user can follow the whole process of Quality Assurance.

The user comprehends the Bologna Process.

The user is able to apply QA in the field of eLearning.

The user can reproduce the different Quality Models, which are relevant for eLearning-applications.

The user will able to evaluate the quality of eLearning courses.

Module A Title: “Quality Assurance in eLearning” Main topics: Overview of Quality Management and Quality Assurance Methods and systems of Quality Management Quality Management in VET and eLearning Process of Quality Assurance iQTool Quality model VET-vocational Education and Training

Title: “Quality Assurance in eLearning”

Main topics:

Overview of Quality Management and Quality Assurance

Methods and systems of Quality Management

Quality Management in VET and eLearning

Process of Quality Assurance

iQTool Quality model

Storyboard Module A 71 screens with: 5 interactive exercises 10 charts 10 animations (approx. 5 - 8 in Flash)‏ study time: 8 hours

71 screens with:

5 interactive exercises

10 charts

10 animations (approx. 5 - 8 in Flash)‏

study time: 8 hours

Module B Title: “User Manual for the iQTool Software” Main Topics: iQTool navigation iQTool user roles iQTool applications (administration, questions and questionnaires, statistics and publishing)‏

Title: “User Manual for the iQTool Software”

Main Topics:

iQTool navigation

iQTool user roles

iQTool applications (administration, questions and questionnaires, statistics and publishing)‏

Storyboard Module B 46 screens with: 14 interactive exercises, 18 screenshots 1 questions for self assessment study time: 2 hours

46 screens with:

14 interactive exercises,

18 screenshots

1 questions for self assessment

study time: 2 hours

Creating questionnaires

WP5 – Testing and Pilot training The two modules: Module A: Quality Assurance in E-learning Module B: User Manual for the iQTool software tool e-Learning SCORM 1.2 compliant modules Each Partner will install 8 e-Learning SCORM 1.2 compliant modules on ILIAS LMS Platform, in their local languages, to test the iQTool tool and the other modules (Module A and Module B), during the test phase

The two modules:

Module A: Quality Assurance in E-learning

Module B: User Manual for the iQTool software tool

e-Learning SCORM 1.2 compliant modules

Each Partner will install 8 e-Learning SCORM 1.2 compliant modules on ILIAS LMS Platform, in their local languages, to test the iQTool tool and the other modules (Module A and Module B), during the test phase

WP6 – some findings All partners works seriously and are committed to achieve the promised results to their best ability. More focus on QA systems in general than on QA for e-learning resources and learning processes. Difficult to glue together the different processes and results in each WP into a common product where one result build on top of the other. Difficult to synchronize activities among the partners. English as working language – misconceptions and different interpretations of the same text arises

All partners works seriously and are committed to achieve the promised results to their best ability.

More focus on QA systems in general than on QA for e-learning resources and learning processes.

Difficult to glue together the different processes and results in each WP into a common product where one result build on top of the other.

Difficult to synchronize activities among the partners.

English as working language – misconceptions and different interpretations of the same text arises

Thank you More information of the project can be found at iQTool web site: iqtool.eu

More information of the project can be found at iQTool web site:

iqtool.eu