Track 11. Gamification Ecosystems Authors: Alberto Mora, Elena Planas and Joan Arnedo-Moreno

1. Designing game-like activities to engage adult learners in
higher education
A. Mora, E. Planas & J. Arnedo-Moreno
Universitat Oberta de Catalunya
{amoraca|eplanash|jarnedo}@uoc.edu
November 2, 2016
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 1 / 24

2. Index
1 Introduction
2 Description
3 Evaluation
4 Discussion
5 Conclusions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 2 / 24

3. Introduction
Highly motivated students have better chances of acquiring
competencies in higher education
Many studies are about how encourage the use of game design
principles for student engagement
Most of them are focused on the 20-25 age bracket in face-to-face
full-time studies
UOC, a completely online-based university
Demography: >30 years old (64%), working full-time (95%)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 3 / 24

4. Background
Gamification: ”the use of game design elements in non-game
contexts” (Deterding et al., 2011)
Educational experiences in the literature present diverse results
It is a delicate design process: ad-hoc versus frameworks (Mora et al.,
2015)
There is not enough evidence about engagement of adults learners in
higher education
Will adult learners be motivated in solving formative activities
with game-like course design?
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 4 / 24

5. Description: foundations
Requirements Engineering (RE) scope
Computer Engineering grade (optional subject)
Study of the activities involved in the requirements engineering
process
Time-line comprised in fifteen school weeks
Evaluation based on three mandatory assignments and
discussions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 5 / 24

6. Description: foundations
Problem to solve
Lack of motivation in solving non-evaluative activities (0% last ed.)
Low sense of comradeship between online students
Expected behaviours
EB1: Motivated students to solve formative (not graded) activities
EB2: Increase of the sense of kinship between e-learners
Motivation types
MT2: Disciplined-materialist students (20.6%)
MT3: Persistent-idealist students (28.8%)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 6 / 24

7. Description: design principles
A design framework that aims to structure the design process
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 7 / 24

8. Description: design principles
SPARC analysis
Sense: coherence with the learning process
Purpose: a clear intention form the instructors standpoint
Autonomy: let students to make choices
Relatedness: positive impact on the rest of students and the course
Competency: assurance that students are able to master the tasks
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 8 / 24

9. Description: the rules
Inspired in a reward-based crowd-funding model
Formative tasks: type (individual/teamwork), challenge and expirable
Phases: on boarding (simple tasks as tutoring), mid-game (regular
task development) and endgame (final tasks to goal achievements).
Task-activation timeline: 5-10 weekly
Expected average performance: 2-3 tasks per student.
All students could benefit even if they have not participated
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 9 / 24

10. Description: the rules
Table: Example of formative tasks
Id Task description
Points
IND
TEAM
CH
EXP
Phase
T1 Convince at least X students to partic-
ipate in the activity
1 x x on boarding
T2 Resolve an exercise proposed in the
learning resources
1 x mid-game
T3 Resolve a doubt posted in the course
forum from another student at most 2
days after posting
1 x x mid-game
T4 Commit to obtain the highest grade in
an evaluative activity of the course
1 x x x mid-game
T5 Resolve an exam model 1 x x mid-game
T6 Correct the resolution of an exam de-
veloped by another student
1 x x mid-game
T7 Answer a survey about the course 2 x end-game
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 10 / 24

11. Description: the rules
Table: Example of rewards
Id Academic-reward Total grade to
be achieved
R1 Have extra days to deliver the evaluative activity being
carried out at the time of reaching the score
10
R2 Evaluate the current activity on 12 points instead of 10 25
R3 Have extra days to deliver the rest of the evaluative ac-
tivities of the course
40
R4 Provide the students with examples of resolved exams 60
R5 Do not take into account the worst exercise during the
evaluation of the exam
80
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 11 / 24

12. Description: the metaphor
An Agile software development project (Scrum)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 12 / 24

13. Description: the tool
Trello, a web-based project management application based on Agile
paradigm
It provides a highly interactive experience (Drag & Drop)
Each task is represented by a card in the board,
Widely accepted in other gamified experiences (Gonz´alez & Mora,
2014)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 13 / 24

14. Evaluation: methodology
Methodology for evaluation inspired in ”Four Level Model”
(Kirkpatrick, 1967): reaction, how learners respond to the
applied process
Remember: the purpose is not the improvement of marks from
previous courses
Instruments: task manager, forum and questionnaires
Managing quantitative & qualitative data
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 14 / 24

15. Evaluation: population (quantitative)
The study was conducted since September 2015 to February 2016
Two groups of the Requirements Engineering course
Demographic, N=94; Age range=22-53; M=34.07; Me=34 and
SD=6.88; 91/94 male
Sample: 60 students enrolled voluntarily into the game-like activity
(63.82% of total)
Representative sample
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 15 / 24

16. Evaluation: performance (quantitative)
First group 62/100 points, second group 80/100
Drop out rate of 0%
26.67% of passive participants (0 task)
18.34% of lower active participants (<2 tasks)
54.99% of higher active participants (>= 2 tasks)
An increment of 35% of posts from previous editions
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 16 / 24

17. Evaluation: growth (quantitative)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 17 / 24

18. Evaluation: correlation analysis (quantitative)
Pearson’s coefficient (age, group, and accomplished tasks)
Positive medium-moderate correlation between ”age of active
participants” and ”accomplished tasks”
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 18 / 24

19. Evaluation: questionnaire (qualitative)
Table: Design questionnaire (n=32)
Question Scope Acceptance
Do you think the tasks have been proposed
make sense in the context of this subject?
Sense 93.75%
Do you think it has been achieved the goal of
encouraging the process of student learning?
Purpose 68.75%
Do you think it has been achieved the goal
of encouraging student participation in activ-
ities?
Purpose 75%
Have you freely chosen tasks you wanted to
develop?
Autonomy 93.75%
Do you think it has been achieved the goal of
promoting teamwork?
Relatedness 56.25%
Do you think that the activity has created
fellowship feeling with other students ?
Relatedness 53.13%
Do you think it has been achieved the goal of
learning about project management tools?
Competency 90.63%
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 19 / 24

20. Discussion (individual overview)
The majority of the course students (63.82%) voluntarily participate
Half of the students revealed to be intrinsically motivated
More than a half of them participated in two or more tasks
(expectation).
Motivating adult learners in solving non-graded formative
activities was reached (EB1)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 20 / 24

21. Discussion (group overview)
No behaviour difference between groups until two weeks before
ending (R4 & R5 achievements)
Appearance of improvised leaders (role of motivators and organizers)
25.70% of the total posts in forum were directly related to the
game-like experience
Increment of the sense of kinship between students, although
to a lesser extent than desired (EB2)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 21 / 24

22. Conclusions
It has been presented the design and analysis of a gameful e-learning
experience with adult on-line learners
SPARC framework was proposed as designed approach
Objective: Improving the student motivation in two different ways
A total of 60 students voluntarily joined the experience
Quantitative and qualitative analysis was conducted
”Relatedness” dimension need further improvement
Further work: personalization of gamification (MT1...MT4)
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 22 / 24

23. References
S. Deterding, R. Khaled, L. E. Nacke, and D. Dixon.
Gamification: Toward a definition.
In CHI 2011 Gamification Workshop Proceedings, pages 12–15, 2011.
A. Mora, D. Riera, C. Gonz´alez, and J. Arnedo-Moreno.
A literature review of gamification design frameworks.
In proceedings of Games and Virtual Worlds for Serious Applications (VS-Games),
2015 7th International Conference on, pages 1–8. IEEE, 2015.
C. Gonzalez and A. Mora.
Methodological proposal for gamification in the computer engineering teaching.
In Computers in Education (SIIE), 2014 International Symposium, pages 29–34.
IEEE, 2014.
D. L. Kirkpatrick.
Evaluation of training.
1967.
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 23 / 24

24. Thank you!!!
A. Mora, E. Planas & J. Arnedo-Moreno Universitat Oberta de Catalunya November 2, 2016 24 / 24