The document describes an eQuality portfolio for evaluating a training program held in Prague from June 21-22, 2012. It discusses several models and methodologies for measuring training quality, including Kirkpatrick's four levels, EFQM, and EFQUEL. Survey results showed high participant satisfaction rates and improved knowledge. While the tools effectively measured quality, the training could be further improved by implementing more activities or extending its duration.

1. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Every Trainee Matters
eQuality Portfolio
Prague, 21th – 22th June 2012
Leonardo Da Vinci - Partnership
George Stefas
Strategic Management

2. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Content
1.eQuality
Portfolio –
Application
Form
3.Known
methodo-
logies
2.eQuality +
4.Statistics +
How to Conclusions
measure

3. Every Trainee Matters / 2010-1-ROI-LE004-0677111
eQuality Portfolio
According to the stated in the Application Form
• Development of an eQuality Portfolio:
“Includes tools & procedures for self-evaluation, for implementing
quality in VET”
Evaluate:
• Quality of the Portfolio
• Degree of satisfaction of the beneficiaries

4. Every Trainee Matters / 2010-1-ROI-LE004-0677111
eQuality Portfolio
2nd axis
3rd axis
4th axis
1st axis
• ePlatform • ePlatform • Evaluation • Final
- functionality Quizzes Questionnai Evaluation
- courses res Questionnai
res (report)

5. Every Trainee Matters / 2010-1-ROI-LE004-0677111
eQuality
Definition
• “the totality of features and characteristics of a product or
service that bears its ability to satisfy stated or implied
needs”*
• Another dimension of quality emanates from the teacher’s
expectations of course quality as the alignment of teaching
tasks, learning activities, and assessment as well as reported
levels of student satisfaction
* ISO 8402-1986 standard

6. Every Trainee Matters / 2010-1-ROI-LE004-0677111
eQuality
How to measure it…
• The evaluation areas that need to be examined for the evaluation of the
quality of a training programme are:
– Material content Knowledge
– Structure /Virtual Environment of quality
– Communication cooperation and interactivity
– Student Assessment
– Flexibility and Adaptability
Analysis
– Support and Quality Experience
criticism of Competence of quality
– Staff qualifications and expertise quality
– Vision and institutional leadership
– Recourse allocation
Design of
quality

7. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Model + Methodology
Kirkpatrick’s Four Levels of Learning Evaluation
EFQM (European Foundation for Quality Management) Model
EFQUEL (European Foundation for quality in e-learning) Model
Open e-Quality learning standards

8. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Kirkpatrick’s Four Levels of Learning
Evaluation
• A sequence of ways to evaluate training programs:
– As you proceed through each level, the evaluation becomes more difficult, requires
more time and provides more information
• The four levels of Kirkpatrick's evaluation model essentially measure:
1. Reaction - what participants thought and felt about the training (satisfaction;
"smile sheets“)
2. Learning - the resulting increase in knowledge and/or skills, and change in
attitudes
3. Behavior - transfer of knowledge, skills, and/or attitudes from classroom to
the job (change in job behavior due to training program)
4. Results - the final results that occurred because of attendance and
participation in a training program (can be monetary, performance-
based, etc.)

9. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Kirkpatrick’s evaluation
Diagram
Evaluation
of results
(transfer or impact to
society)
Evaluation of behavior
(transfer of learning to workplace)
Evaluation of learning
(knowledge or skills acquired)
Evaluation of reaction
(satisfaction or happiness)

10. Every Trainee Matters / 2010-1-ROI-LE004-0677111
EFQM Model
(European Foundation for Quality Management)
• A model of comprehensive quality
management, 1991, popular in Europe
• Assessment of performance & progress and helps
organizations in continuous improvement activities
• Advantage:
– Direct relation to the European context
– Focuses on the defined objectives
• 9 Criteria:
– 5 Enablers: what an organisation does
– 4 Results: what an organisation achieves
Enablers cause Results and Results improve Enablers

11. Every Trainee Matters / 2010-1-ROI-LE004-0677111
The EFQM Model
Diagram

12. Every Trainee Matters / 2010-1-ROI-LE004-0677111
EFQUEL Model
(European Foundation for quality in e-learning)
• Main Phases for the implementation of eQuality Platform:
– Managing & Implementing e-Quality
– Vision & Strategy planning
– Encouraging individual & organisational learning through e-Quality
• Each phase has 5 stages:
– Not yet started
– Early stages/ Awareness
– Developing/Commitment
– Established
– Embedded/Advanced

13. Every Trainee Matters / 2010-1-ROI-LE004-0677111
The EFQUEL Model
Diagram
2. Vision and Strategy planning
1. Managing and Implementing e-Quality
1.1 e-Quality 2.1 Clarity of vision
learning through eQuality
3. Encouraging individual and organisational
3.1 Managing staff
platform for the development competencies
implementation of eQuality within the development with ICT
organisation
1.2 e-Quality strategy 3.2 Recording
planning and 2.2 Integration of individual progress
implementation eQuality strategy in and achievement
other strategic plans with ICT
1.3 Partnerships
2.3 Strategic 3.3 Sharing
1.4 Monitoring and
management of knowledge across
reviewing
eQuality within the groups and
implementation
organisation communities with ICT
1.5 Sustainability
and return on
investment

14. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Open e-Quality learning standards
• Open e-Quality Learning standards developed by Learning Innovations
Forum d’Innovationsd’Apprentissage (LIfIA) in the Americas and the
European Institute for E-Learning (EIfEL) in the EU
• e-Quality standards describe the learning objectives in terms of
knowledge and skills gained after this training process
• Blended learning:
– The student / learner acquire specific content skills and knowledge
– The student / learner acquire learning skills
– Self-directed learning management
– The student is awarded course credits/credentials
– The learner gains an adequate Return on Investment (RoI)

15. Every Trainee Matters / 2010-1-ROI-LE004-0677111
• Measureable intended learning outcomes
• Accurate curriculum content
• Adaptable teaching / learning material for learners needs
• Clear product / service information for potential students
• Use of appropriate Learning technologies

16. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Effectiveness of the elearning process
Platform Quizzes – participants scores
Organization Overall grades of all participants
ExpertOnes S.A. 68.84
FCIP-GIP de Bourgogne 69.84
Future-NGO 54.60
CVT-SBIE 58.79
SWSPiZ 58.40
IELM 74.21
* ITC N.A.

17. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Evaluation of the training process
Evaluation Questionnaires – participants view of the training process
• 84% are satisfied with the elearning tools used
• 99% rated above average of the instructor’s knowledge of the subject matter
• 93% declared that the trainers were able to relate the knowledge of the training material to real world
experiences and situations
• 96% of them rated their performance of the instructor above average
• 72% rated good the clarity of the training content
• 78% acknowledged that this training helped them increase their knowledge and skills
• 75% of participants estimated that their participation in the training programme, will have a positive
impact on their job effectiveness
• 69% of the stakeholders estimated that their participation in this training programme will affect positively
their career potential
• Despite the fact that almost 84% of the respondents rated the learning effectiveness of this training
programme positively, only 5% estimated that this training approach was extremely effective. This fact
indicates that the training programme should have implemented more activities or maybe conducted in a
longer time period.
• 84% stated that there were satisfied with this training

18. Every Trainee Matters / 2010-1-ROI-LE004-0677111
Conclusions
• The tools and methodologies implemented were sufficient in measuring the qualitative
aspects of the training programme
• ePlatform developed under the elearning Quality models and standards used at worldwide
level
• The results demonstrating that the learners’ comprehension (quizzes) was above average,
thus achieving the main learning objective: the comprehension of the learning material
• The results evaluating quantitative and qualitative aspects of the training by the trainees
demonstrated that overall was qualitative and successful
– The learners estimated that they had gained new knowledge from the learning programme and the
majority of them were eager to recommend this training approach to other individuals.
• Through the results the training process, the Portfolio, the training material, the ePlatform
can all be improved under the quality perspective.