The document outlines a four-level framework for assessing quality practices in active e-learning, detailing key concepts like learner-centeredness, relevant technology use, and planned alignment in e-learning design. It emphasizes the importance of engagement, collaboration, and authentic learning experiences to enhance student involvement. The framework consists of various sub-dimensions and categories that aid in evaluating institutional and participant backgrounds, program design, and the learning process.

1. A four levels framework to understand
quality practices in active e-Learning
Diogo Casanova1, António Moreira2, Nilza Costa2
1 Kingston University London, CHERP, United Kingdom
2 Universidade de Aveiro, Portugal
ECER2014

4. How are technologies being used in learning and
teaching? A view of the sector in the UK
60%
50%
40%
30%
20%
10%
0%
Sector mean 2003
Sector mean 2005
Sector mean 2008
Sector mean 2010
Sector mean 2012
Sector mean 2014
web supplemented, in which
online participation is optional
for students
web dependent (interaction
with content)
web dependent
(communication with
staff/students)
web dependent (interaction
with content and
communication
fully online courses
Universities and Colleges Information Systems Association

5. What does the Horizon report say (2014)
REASONS THAT ARE IMPEDING TEL ADOPTION IN HIGHER EDUCATION
The low digital fluency among staff
• the lack of competencies in using technologies pedagogically;
• the non-integration of digital media literacy skills across the curriculum.
The lack of rewards and recognition for good teaching practices and
especially online teaching, which is still considered to be an extension to the f2f
traditional environments (in campus-based universities)
The emergency of new models of education such as MOOCs,
that bring new pedagogical challenges when there is still a lot to learn
about delivering effective online education

6. 2003
f2f E-Learning
2014
Pedagogy
Passive
f2f E-Learning
Pedagogy
Active
Supplement
Active learning
What are the repercussions for quality?
Where are we now?

8. oWhat is active e-Learning?
oAre we prepared for engaging in active e-
Learning… are our students… is our institution?
oWhat is quality in active e-Learning?

10. research design
1. Theoretical
study
2. Empirical
study
4. Discussing
and reflecting
on data
3. Collecting,
and
developing
instances and
categories
5. Theoretical
study
6. Empirical
study
8. Discussing
and reflecting
on data
7. Collecting,
and
developing
instances and
categories

11. First phase (focusing on the definition of active e-Learning)
Step - 1 data collected from the theory, especially focusing on theories of e-
Learning and active learning;
Step - 2 interviews with members of academic and non-academic staff from a
university in Portugal;
Step - 3 data collected from the theory and empirical findings and formulation of
instances and categories within the framework;
Step - 4 data findings discussions with international experts in this field of
knowledge and elaboration of a revisited framework.

12. Second phase (focusing on identifying QS for active e-Learning)
Step – 5 data collected within the literature about models that evaluate the quality
of e-learning which in some extent would be aligned with the first version of the
framework;
Step – 6 focus group for understanding the students point a view about quality
learning in e-Learning,
Step - 7 data collected from the theory and empirical moments and formulation of
concepts and categories within the framework;
Step - 8 data findings discussion both in Portugal and in the UK with international
experts in e-Learning and e-Learning quality.

14. Key concepts
learner-centred
The learner should be able to act as an active agent in the learning process
designing its own path of learning and constructing its own conception of
knowledge (Mayes & Freitas, 2007).
Should allow a degree of personalisation either by agreeing with learners the
learning outcomes and assessment or by giving learners different opportunities
so they can adapt to their own learning styles and needs.
Examples:
• Allow learners to choose their own learning resources and materials;
• Discussing personal experiences and setting individual outcomes/goals;
• Enforcing learners to become active consumers of online communities.

15. Key concepts
relevant use of technology
Technology must be used when it is relevant and when it brings something
new or additional to the learning experience.
This suggests that by using technology the lecturer is either promoting new
learning opportunities or promoting more effective and more engaging
teaching.
Examples:
• Use technology to promote asynchronous and reflected discussions;
• Use technologies to promote collaborative writing (Google Docs) or thinking skills
(Conceptual Maps);
• Use technologies for interacting with students in classroom (Twitter wall, Audience Response
Systems) or involving with a different style of learning using multimedia files or simulation
environments.

16. Key concepts
planned and designed
E-Learning doesn’t match with improvise. Activities need to be aligned with assessment, with
iLO and with a pedagogical goal.
Practical use of constructive alignment (Biggs & Tang, 2011) is fundamental because it
allows learners to become more autonomous and more responsible for their learning.
Lecturers should choose a particular tool or technology according to the learning activity or
to each pedagogical goal (Conole, Dyke, Oliver, & Seale, 2004; Laurillard, 2002).
Examples:
• Use design tools to plan and to reflect on how you set learning goals and plan activities;
• Use analytics as a tool to reflect and redesign.

17. Key concepts
active and experiential
The learner has to become involved in the learning process not by hearing or reading
passively but by appropriating the content delivered. Learners must be involved in
developing hypothesis, interpreting, manipulating, solving problems and taking
decisions (Kim & Hannafin, 2011).
The learner must also be involved in reflecting on its learning and on the learning
process.
Examples:
• Develop project base learning activities (technology can be seen as a medium but also as a
support for the end product);
• Promote complex activities which suggest experimentation and having different outcomes;
• Encourage individual and group reflections at the end of each activity (using blogs, social
networks or forums).

18. Key concepts
flexible
One of the most suggested reasons for the use of technology in
learning and especially in HE is the degree of flexibility it allows, not
just from the learners perspective but also from the lecturer
perspective.
Flexibility can be demonstrated according to the time learning
occurs, the duration learning takes, the learning space or the
learning resources available (Moore et al., 2011; Oliver & Trigwell,
2005).
Examples:
• Agree with flexible schedules (tasks, assessment) when possible;
• Encourage flexibility when choosing resources or tools;

19. Key concepts
engagement by production and collaboration
For the use of technology to become engaging it should foster production, collaboration,
communication and content sharing between lecturers and learners and learners and
learners (Conole, Crew, Oliver, & Harvey, 2001; Shea, Pickett, & Pelz, 2003).
Communication needs to be effective and according to best practices (netiquette, frequent,
relevant). There should be a presence from the lecturer either directly or indirectly
(mentors/tutors) and communication must be synchronous and asynchronous .
Feedback must be constructive, timely and involving learners in it (Shea et al., 2003).
Examples:
• Nominate students as moderators so they become daily responsible for the online environment
(fosters responsibility, transferable skills, online presence);
• Agree with a response time beforehand and give feedback in that timeframe (ideally in a 24h
timeframe).

20. Key concepts
authentic
Alignment between activities, students expectations and experiential learning
(Bradwell, 2009; Herrington, Oliver, & Reeves, 2011). A learner motivated
with real and meaningful activities is crucial especially when there is a lack of
physical presence from the lecturer. It encourages engagement and self-direction.
Examples:
• Design real scenarios similar to what students would have in their future professions;
• Encourage more project base learning;
• Involve stakeholders/employers when possible;
• Encourage students to share their previous experiences;
• Encourage the use of professional networks (social media).

21. other relevant concepts that
informed the framework
Innovation (materials and resources, learning activities)
Impact (students learning, transferable skills)
Satisfaction (students, staff, future employers)
Sustainability (lifelong learning skills, reusable materials
and resources)

23. The four level model
12 sub-dimensions
61 domains
121 categories
298 threshold statements
Level 4 -
Learning
process
Level 1 -
Institutional
background
Level 3 -
Design
Level 2 -
Participants
background

24. Level 1 –
Institutional
background
Information systems
Institutional policies
Support
Guidelines and procedures
Evaluation
Quality
Technical
Pedagogical
Librarian
Administrative

25. Level 2 -
Participants
background
Competencies
Scientific
Pedagogical
Technical
Motivational
Digital
Confidence
Constraints
Expectations
Requirements
Convergence of roles
Convergence of users expectations

26. Level 3 - Design
Programme attributes
Learning content
Learning activities
Information about content
Information about rules
Information about requisites
Alignment with the curriculum
Flexible
Accurate
Learner-centred
Aligned with the market
Level of technology integration
Level of participation
Level of active learning

27. Level 4 – Learning
process
Interpersonal relationship
Delivery
Communication
Assessment
Learning resources
Instruction
Pedagogical domain
Immersion domain
Technical domain
Efficacy domain
Scientific domain
Learning Environment
Pedagogical domain
Technical domain
Organisational domain
Immersion domain
Aesthetic domain

28. Final considerations and suggestions for discussion
• Is this module more suitable for quality enhancement or
quality assurance? What concerns should we have?
• Can we use this module for assessing emergent e-learning
trends (MOOCs, OER, PLEs)
• How and with what debt should we integrate analytics in
evaluation models (ethics, data protection).