1.
Enhancing ICT Education through Formative Assessment, Learning Analytics and Gamification
Robert Clarisó, Santi Caballé, Daniel Riera, Jordi Conesa, Isabel Guitart, Àngels Rius, Elena Planas,
David Bañeres, Enric Mor,Teresa Sancho, Ana Guerrero, Elena Rodríguez, MúrielGarcia,
Francesc Santanach, Jordi Casamajó, Ramon Masià
Dept. of IT, Multimedia andTelecommunication
Dept. of EducationalTechnology
Open Universityof Catalonia
Rambla Poblenou, 156. 08018 Barcelona, Spain
Corresponding author’s email: scaballe@uoc.edu
Extended abstract
This abstract presents a new research project in the field of e-assessment named “Enhancing ICT
education through Formative assessment, Learning Analytics and Gamification” (ICT-FLAG) funded bythe
Spanish Government. ICT-FLAG is a three-year project coordinated at the Open University of Catalonia
(UOC). We present next the scope, objectives, research methodology and expected impacts of the project.
Education in the field of Information and Communication Technologies (ICT) includes very practical
competencies, which can only be acquired by means of experience, performing exercises, designs,
projects, etc. In addition to the challenge of motivating students to solve activities, lecturers face the
problem to assess and provide suitable feedback to each submission. Receiving immediate and
continuous feedback can facilitate the acquisition of the competencies, although this requires support in
the form of automatic tools. The automation of the assessment process may be simple in some activities
(e.g., practical activities on programming) but it maybe complex in activities about design ormodeling.
Monitoring the use of these tools can reveal very valuable information for the tracking, management and
continuous improvement of the course bythe teaching team. However, in order to leverage all its potential,
this information should be complemented with data from other sources (e.g., the student’s academic file)
and historical information of previous editions.
To this end, the ICT-FLAG project considers formative assessment (e-assessment) as an appropriate
teaching strategy for learning procedure competences, such as those in the scope of ICT. Formative
assessment is already in practice with success in a variety of teaching areas within ICT education.
However, there are certain teaching areas where it is hard to define an automatic assessment and provide
complex informative feedback.To this end, the project faces the challenge of applying specific techniques
based on formalmethods and artificial intelligence in order to design algorithms for automatic feedback as
well as improve the formative assessment fed back to students.
In addition, the project handles the relevant purpose of combining formative assessment tools with two
more axes: learning analytics and gamification (see Fig. 1). Previous studies explore all these axes
separately, thus having less impact in the learning process. Therefore, the integration of these approaches
willcreate beneficial synergies to all the project’s dimensions.
Considering all the above, the main objective of the ICT-FLAG project is to design and build a set of e-
Learning tools and services to provide support to the learning process in university degrees in the field of
ICT.This general objective is organized in three perspectives:
• From the methodological perspective, the design, development and evaluation of specific tools will
support formative assessment in ICT degrees with the aim of fostering students to perform activities
by an immediate and informative feedback that improves the learning process from right and wrong
concepts. In addition, the design, development and evaluation of specific services will allow for
collecting, analyzing and correctlypresenting relevant knowledge to all the stakeholders related to the
use of formative assessment tools and other key aspects of the learning processes. Finally, by

2.
selecting, defining, installing and evaluating gamification-based mechanisms will increase students’
motivation and engagement in ICT degrees and courses, with an emphasis of the use of the above
mentioned formative assessment tools.
• From the technological perspective, the e-Learning tools developed in this project will consider the
modularityand independence from e-Learning platforms and particular virtual campuses, with the aim
to facilitate its application to different courses and contexts. In addition, the functionalities of the tools
will be offered as a set of services, using appropriate standards. To this end, the outcomes and tools
developed in the project will be independent from specific technological platforms and will separate
the dependent and independent concerns from the implementation technology as well as will foster
the use of appropriate standards. Finally, these tools will be published using open source licenses
and open APIs in order to be at the disposal of students, educators, developers and general public.
• From the exploitation and dissemination perspectives, the project’s outcomes will improve the learning
performance while decreasing the drop-out rate in ICT degrees. In addition, this project will endow
tutors and academic managers with resources to support the monitoring, analysis and decision-
making processes in their degrees and courses while guaranteeing a correct transference and
application of the above contributions to a variety of tools, technological platforms and educational
institutions. In particular, it is expected that the use of the produced tools will become feasible as part
of both self-taught education (life-long learning) and traditional formal education as well as massive
courses of on-line learning (MOOCs). Finally, the above outcomes will be shared with the research
communityand public at large through presentations, publications and outreach activities.
Figure 1. General view of the ICT-FLAG project.
The researchmethodologyof this project will focus on thementioned three axes (see Fig. 1):
• Tools for formative assessment, which can provide immediate feedback by means of automatic
assessment. In particular, the research activity will focus on knowledge areas with high cognitive or
modeling levels, such as the design ormodeling of software and hardware.
• Learning analytics that monitor the activity and the progress of the student about the use of the
mentioned tools and allow for analyzing the learning results, identifying the critical points and defining
actions of improvement. These analytics will also incorporate other sources of academic and historical
information to facilitate the course tracking and decision making processes to the teaching team.
• Gamification as an incentive scheme in order to motivate students to perform new activities and
increase their engagement without sacrificing the academic rigor.
Moreover, specific indicators will be designed to measure the impact in the learning process from each of
the mentioned three perspectives. These indicators will be evaluated from pilot tests during an academic
term at the UOC. In particular, these pilot tests will be run in courses of mathematics, computing
engineering and telecommunication given at the UOC.

3.
The expected benefits of the project will have a relevant impact on students, lecturers, managers and
academic coordinators, being societyas a whole an indirect beneficiary. In particular, the expected impact
for students is the improvement of the educational experience and engagement during the practical
activities, which will enhance learning from mistakes by an informative and immediate feedback. This
impact will result in students’ greater participation and performance while lowering drop-out rates and
improving satisfaction and retention levels.
Furthermore, tutors, academic coordinators and managers will be provided with open source tools that will
facilitate the formative assessment and feedback processes. For instance, tutors will have resources for
monitoring and analysis of a course for making decisions and improving the teaching quality, resulting in a
better management of certain courses with greater attendance, such as those open massive courses
(MOOC). As a result, educational institutions will perceive these expected impacts as an improvement of
the academic outcomes, better students’ satisfaction with the ICT courses and provision of complete
information on the students’ activities for decision making from prediction and estimation. Society at large
will leverage the generalization of this approach in the long term by an improvement of the perception of
ICT courses and a more efficient ICT higher education, resulting in a better response to the great demand
of ICT professionals.
Finally, from the scientific and technical perspectives, the project willcontribute to the advance of the state
of the art and provide better understanding of the different problems and challenges of current e-Learning
and general education. In particular, the project will address the application of automatic techniques for
assessment of ICT learning activities, strategies to provide immediate and useful feedback to students’
activities, methods to collect, analyse and correctly present the information and extracted knowledge in
educational environments, and the application, benefits and challenges of using gamification techniques in
an academic context. The outcomes generated from this research will be addressed from both theoretical
and practical views, including experiences with open source tools that will allow other educational
institutions and organizations to apply, evaluate and reproduce the ICT-FLAG project’s contributions.
Acknowledgements
This research was partly funded by the Spanish Government through the project TIN2013-45303-P "ICT-
FLAG" Enhancing ICT education through Formative assessment, Learning Analytics and Gamification.
References
Ihantola, P., Ahoniemi, T., Karavirta, V., & Seppälä, O. (2010). Review of recent systems for automatic
assessment of programming assignments. In Proc. Koli calling Conf. on Computing Education Research,
86-93. ACM.
Romero, C., Ventura, S. (2010). Educational Data Mining: A Review of the State of the Art. In IEEE Tran.
on Systems, Man, and Cybernetics, Part C 40(6), 601-618.
Ferguson, R. (2012). Learning analytics: drivers, developments and challenges. Int. Journal of Technology
Enhanced Learning, 4(5/6), 304–317.
DomínguezA., Saenz-de-Navarrete J., de-Marcos L., Fernández-SanzL., Pagés C., Martínez-HerráizJ.J.
(2013). Gamifying learning experiences: Practical implications and outcomes, Computers & Education, 63,
380-392.
Siemens, G. (2013). Massive Open Online Courses: Innovation in Education? Commonwealth of learning,
Perspectives on Open and Distance Learning: Open Educational Resources: Innovation, Research and
Practice, 5. ,2013.