Track 3. Evaluation in education and guidance Authors: Jaione Cubero-Ibáñez, María Soledad Ibarra-Sáiz and Gregorio Rodríguez-Gómez

1. Development and use of mobile technologies
that foster students' evaluative judgement: a
design-based research
Jaione Cubero Ibáñez, María Soledad Ibarra-Sáiz, Gregorio Rodríguez-Gómez
EVALfor Research Group - University of Cádiz
Spain
Sixth Edition Technological Ecosystems for
Enhancing Multiculturality
24-26 October 2018
Salamanca, Spain

2. Objective
To verify the results and the impact obtained from the use of
mobile technologies in the development of evaluative
judgement in participatory assessment processes as a means
for the training of democratic and critical citizens.

3. Theoretical Framework
(Rodríguez-Gómez & Ibarra-Sáiz, 2015)

4. Students’ evaluative judgement
Evaluative judgement
“the ability to make decisions about the quality of work of self and others”
(Tai, Ajjawi, Boud, Dawson, & Panadero, 2018, p. 471)
Components:
• understanding what constitutes 'quality‘
• applying this understanding through an assessment of the work,
whether one's own or that of another person
to provide opportunities for students to exercise
and develop their evaluative judgement

5. Mobile learning
“the processes (both personal and public) of gaining knowledge via
exploration and conversation through multiple contexts between people and
interactive technologies“
Adams Becker, S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C., and Ananthanarayanan,
V. (2017)
A Model for Framing Mobile Learning (Koole, 2009)

6. Operational goals:
1. To develop a Conceptual Framework for the study
2. To design, programme and develop mobile technologies
that streamline the implementation of students' evaluative judgement
in participatory assessment processes.
3. To experimentally apply these mobile technologies in different
subjects within a framework of assessment as learning and
empowerment.
4. To evaluate the effects (results and impact) of student participation in
assessment by using mobile technologies.

7. Methodology
Design-based research methodology (DBR)
(Anderson & Shattuck, 2012)
• it is situated in a real educational context
• it is focussed on the design and verification of a meaningful intervention
• it uses mixed methods
• it involves a series of iterations (at least two cycles) in which
researchers and practitioners work together

8. Intervention design

9. Causal model of relationship between variables

10. Iterative cycles based on DBR methodology

11. Master or Degree Module n
Business Administration (UCA) Project Management 135
Finance and Accounting (UCA) Project Management 25
Early Childhood Education (UCA) Educational Innovation and Research 150
Education (USAL)
Assessment Techniques and Instruments 60
Educational Guidance 60
Master in Secondary Education (USAL)
Educational Guidance 20
Career Guidance 20
Business Administration (URIV) Economics and Business Management 50
Total 520
Sample

12. Data collection
Information sources
Data collection
techniques
Data collection instruments
Tutors Observation
 Tutor Diary
 Critical incident register
Students Survey
 Initial competence level pre-test questionnaire
 Final competence level post-test questionnaire
 Questionnaire about the satisfaction and
usefulness of the designed applications
 Final group interview

13. Design Implementation Evaluation
Phases

14. Expected Outcomes
 Providing mobile technologies that enable student participation in
the assessment process
 Increasing the level of university students’ evaluative judgement
skills
 Increasing the degree of university-based innovation in coherence
with today's digital context
 Educating participative, critical, committed students who transfer
these skills to a real-life context and contribute to the consolidation
of an inclusive and reflective society

15. Development and use of mobile technologies that foster students’
evaluative judgement: A design-based research
TEEM 2018 Sixth Edition Technological Ecosystems for Enhancing Multiculturality
24 – 26 October, Salamanca, Spain
© Jaione Cubero-Ibáñez
María Soledad Ibarra-Sáiz
Gregorio Rodríguez-Gómez
Attribution-NonCommercial-NoDerivs CC BY-NC-ND
EVALfor Research Group
University of Cádiz