Track 3. Evaluation in education and guidance Authors: Mikel Villamañe, Ainhoa Alvarez, Mikel Larrañaga, Oscar Hernández-Rivas and Jessica Caballero.

1. TEEM 2018 – 25th October 2018
TEEM 2018 – 25th October 2018
Using Visualizations to Improve Assessment in
Blended Learning Environments
Mikel Villamañe, Ainhoa Álvarez, Mikel Larrañaga,
Oscar Hernández-Rivas, Jessica Caballero
University of the Basque Country UPV/EHU

2. TEEM 2018 – 25th October 2018
• Introduction
• COBLE
– Knowledge Model
– Data management module
– Feedback module
• Acceptance evaluation
• Conclusions & future work
Agenda
2

3. TEEM 2018 – 25th October 2018
• Assessment is a key element in teaching/learning
processes
• A good assessment process entails analyzing
students’ results
– To detect anomalies and provide feedback to solve
them
Introduction
3

4. TEEM 2018 – 25th October 2018
• Assessment information can come from different
sources. Moreover in blended-learning
environments
• This data heterogeneity makes difficult to extract
valuable information
• Tools that facilitate to aggregate and integrate
information from different sources are required
Introduction
4
…

5. TEEM 2018 – 25th October 2018
Competence-Based Learning Environment
COBLE
5

6. TEEM 2018 – 25th October 2018
COBLE
6
……
KNOWLEDGE MODEL
DATA MANAGEMENT MODULE
FEEDBACK MODULE

7. TEEM 2018 – 25th October 2018
• Domain knowledge
– Course structure: competences, topics, assessable
items...
• Student knowledge
– Historical data: information regarding work sessions
– Knowledge level
– Competences Mastery level
COBLE – Knowledge Model
7

8. TEEM 2018 – 25th October 2018
COBLE – Data Management Module
8
a
b
c
d
e
KNOWLEDGE MODEL
Historical data Knowledge level Mastery level

9. TEEM 2018 – 25th October 2018
• Provide lecturers relevant visual information
– To track their students
– Refine teaching and assessment practices
accordingly
COBLE – Feedback Module
9

10. TEEM 2018 – 25th October 2018 10
COBLE – Feedback Module

11. TEEM 2018 – 25th October 2018
• Acceptance is critical for the inclusion of any
ICT solution
• Unified Theory of Acceptance and Use of
Technology (UTAUT)
• 8 lecturers (4 women, 4 men) with more than
10 years of experience
• 30 minute session
Acceptance evaluation
11
Introduction of
functionalities
Free interaction
with the system
Interview

12. TEEM 2018 – 25th October 2018
• Users found COBLE useful
• The system would be helpful to detect
problematic situations in the assessment
process.
• 62,5% stated they would like to use the system
Acceptance evaluation
12
I intend to use the system
next academic year
Yes Yes, but integrated I don't know

13. TEEM 2018 – 25th October 2018
• Conclusions
– COBLE helps lecturers to integrate the information
gathered from different sources
– COBLE also provides various visualizations that can
help teachers to detect anomalous situations.
– COBLE has been tested in a real scenario with
lecturers from diverse courses
Conclusions & future work
13

14. TEEM 2018 – 25th October 2018
• Future work
– Use COBLE in several courses to evaluate whether
the use of the system has any effects on the
course´s instructional design and on the students’
learning process.
– Provide the students access to the visualizations
– Include for the teachers the possibility of sending
messages to students
Conclusions & future work
14

15. TEEM 2018 – 25th October 2018
TEEM 2018 – 25th October 2018
Using Visualizations to Improve Assessment in
Blended Learning Environments
THANK YOU
Contact: ainhoa.alvarez@ehu.eus
http://galan.ehu.eus