The document summarizes evaluations from students in Norway on a mobile app called MILAGE that was used as part of a blended learning project for teaching mathematics. Students found the app to be interesting, easy to use, and innovative. They believed it supported individual and collaborative work. Most students indicated that instructional videos were the most helpful feature for learning. Overall, the evaluations found that students were actively engaged with constructing content for the app and viewed it as a useful additional approach for learning mathematics.

1. Blended Learning With Mobile Technology
Beata Godejord
Per Arne Godejord

2. MILAGE – Mathematics bLended Augmented GamE
Duration: 36 months (01.09.15 – 31.08.18)
Full title:
Project Acronym:
Project website: http://milage.ualg.pt
Key Action 2 – Cooperation for Innovation and the Exchange of Good Practices
Action - Strategic Partnership for school education
Interactive Mathematics by implementing a Blended-Learning model with
Augmented Reality and Game books
MILAGE - MathematIcs bLended Augmented GamE – supported by European ERASMUS+
Project reference: 2015-1-PT01-KA201-012921
Introduction

3. Partners:
University of Algarve
University of Extremadura
Çağ University
Portugal / Project Coordinator Norway
Nord University
Spain Turkey
Escola Secundária Pinheiro e Rosa Verdal videregående skole
IES Norba Caesarina
MILAGE - MathematIcs bLended Augmented GamE – supported by European ERASMUS+
Project reference: 2015-1-PT01-KA201-012921
To produce an interactive and motivating learning resource for assisting
the process of teaching/learning mathematics in upper secondary schools
Goal:
Introduction
To determine the learning potential of blended learning
in upper-secondary schools
Using digital media
for expanding
learning opportunities

4. Encouraging students engagement in learning
Active Learning
Mobile Learning
Students using mobile app
for learning mathematics

5. Pedagogical perspective
Production-centered activities
Peer-supported activities
Personalised activities
Implementing digital media
to create an environment that supports
active learning

6. Digital media
MILAGE Learn+ for Teachers
MILAGE Learn+ for Students
eBook
Mobile app Process
Outcome
Social Network
Edmodo
Developmental work
Personal
computers
Video screen
capturing software
Students:
producing, sharing
Students:
using, interacting
Input device
enabling users to hand-draw
mathematical formulas

7. Qualitative evaluation – 1st year of project realization
Objective:
To explore participants’ perceptions, feelings and experiences
of project activities conducted in the first year of project realization
The use of digital media
Videos showing solutions to mathematical problems and media used to produce them

8. Method
Thematic content analysis
Focus groups Gathering qualitative data
Method of analysis
The perspective of constructivist grounded theory
CAQDAS (NVivo11)
Data analysis Inductive (data-driven )
Coding

9. Qualitative evaluation – 1st year of project realization
Hierarchical chart generated by CAQDAS.
The size of the fields area reflects the number of items coded in Norway
Dominant
theme

10. Qualitative evaluation – 1st year of project realization
Perception Process of interpreting
The perceiver
The target
The situational
context
«Technological element»
introduced
to educational process
Interpretation and addition
of meaning to the target
How did the students interpret the «target» and what meaning did they add to it?

11. «The Perceptions of Project Activities»
Qualitative evaluation after 1st year of project realization
The project activities were categorized as a useful initiative that helps learning and enhances the learning process.
Brunner’s model of the perceptual process and an example
Results

12. Representative quotes from students
Norway:
Spain:
Portugal:
Results
“Also, the use of other aids not just book and pencil makes a positive contribution to learning
as we like digital tools and are used to them”.
“We learn in a different way and we learn by doing concrete things, which we can share with others
and we can help others in this way”.
“It helps practicing outside the classroom. In the book, you have exercises and solutions.
With the videos, you can see step by step how to solve the problem.
It is like you were in the classroom.”
“But when you record the solution, firstly you make a structure in your mind or on a paper,
what information you are going to start with and follow. In this process you are learning.”
“Everyone is involved in teaching.”

13. The presented study was guided by quality criteria for digital learning resources developed by
the Norwegian Centre for ICT in Education
as an aid for teachers and others who want to evaluate the sustainability of digital learning resources in educational context.
The objective of the research was to obtain an insight into students’ opinions on a mobile app as a learning resource
and
thus contribute to the discussion on the potentialities of mobile learning and the design of mobile learning applications.
MILAGE - MathematIcs bLended Augmented GamE – supported by European ERASMUS+
Project reference: 2015-1-PT01-KA201-012921
The Norwegian Centre for ICT in Education
https://iktsenteret.no/english
Survey conducted in Norway at the end of the 2nd year of project realization

14. The evaluation criteria focused on three broad categories
MILAGE - MathematIcs bLended Augmented GamE – supported by European ERASMUS+
Project reference: 2015-1-PT01-KA201-012921
User dimension
Distinctiveness of the
digital resource
Subject and education
dimension
The interface between
the user and the resource
The possibilities and limitations
of the digital resource
The educational and
evaluation potential

15. Data were collected through online survey containing:
- Multiple choice questions (2)
- Likert scale questions and (7)
- Open-ended questions (7)
MILAGE - MathematIcs bLended Augmented GamE – supported by European ERASMUS+
Project reference: 2015-1-PT01-KA201-012921

16. User dimension
Does the digital resource create interest?
Results

17. What is interesting about the app?
(15 answers)
User dimension
- «Interesting with digital tool, change in the way we usually work»
- «New way of learning»
- «Use of technology for learning»
- «It’s fun to use MILAGE app because it’s practical work»
- «Seeing what we do wrong and learning from it»
- «We see how others solve the tasks»
- «This is a new type of app, which I haven’t seen before»
- «It’s good to be able to use the smart phone»
- «That it gives us instructional videoes»
- «Learning methods»
- «Different than usual teaching»
- «You may compete with your co-students»
- «New learning method, not much used»
- «New way of learning»

18. User dimension
Is the digital resource inclusive and accessible?

19. User dimension
What makes the app easy or difficult to use?
(15 answers)
- «Straightforward»
- «It’s easy to understand, has good tasks»
- «It’s ordelig and easy to navigate»
- «Very understandable»
- «Easy to understand»
- «The design of the app makes it easy to use»
- «It has to be clearer to show next step»
- «We can’t redo the tasks if we make a mistake»
- «It’s easier to just use the book»
- «It freezes occasionally. Otherwise it has good navigation»
- «It’s a bit weak showing info and how to use the app»

20. User dimension What digital skills does the app support?
(3 answers)
- «I have to have knowledge about digital competence»
- «All»
- «Taking picture»

21. Distinctiveness of the
digital resource
How does the digital learning enable new possibilities that are lacking in traditional learning resources?
Highest score of «Yes» answers

22. Distinctiveness of the
digital resource
What makes the app innovative?
(13 answers)
- «I havent’ seen anything like it before»
- «You learn through an app that is good»
- «We use new methods that we do not use in other courses»
- «I learn a lot by seeing what others have done and what I have done»
- «Taking a picture when finished with the task»
- «Less books, more technology»
- «We may compare our own calculations with the correct answer immediately»
- «A fine way of doing something else»
- «You learn in a diferent way from just using a book»
- «I haven’t heard about such app before, therefore I call it innovative»
- «The app appeal to young people as we are involved with technology»
- «Not many apps that are like this one»
- «It’s innovative because it’s not just book»

23. Subject and education
dimension
Does the digital learning resource enable an evaluation that is adapted to the education setting?

24. Subject and education
dimension
In what way does the app help you reflect on mathematical tasks?
(7 answers)
- «I can see where I made mistake, if any»
- «One can take pictures, etc.»
- «I reflect on the various solutions that exists for each tasks»
- «I can see what mistakes I make in the calculations»
- «I have to do the task correctly in order to take a picture and send it»
- «You repeat the material»
- «We can watch the videoes of how to solve the tasks»

25. Subject and education
dimension
In which educational setting is the digital resource suitable?

26. Subject and education
dimension
In what way does the app support individual work?
(7 answers)
- «I have possibility to work alone before other students/teacher assess my results»
- «I’m developing my skills»
- «I get points»
- «I work more independently»
- «I’m learning by my mistakes»
- «I’m revising the material»

27. Subject and education
dimension
In what way does app support collaborative work?
(11 answers)
- «You may work togethr with others and also see how others solve the task
- «We can collaborate»
- «You may contact people via the app»
- «We kan discuss with each other»
- «Chat» (2)
- «I cooperate with others by making videoes»
- «I can give points to others»
- «We learn from each other by the possiblility of seeing how others solved the tasks»

28. Subject and education
dimension

29. Summary
- Absence of strong «No»
- Prevailance of «Yes»
- The highest amount of «Yes» and «Strong Yes»
- The lowest amount of «Yes» and «Strong Yes»
Likert scale questions

30. Summary
Open-ended questions
The lowest number of responses came to the question about the relation between the use of app and
the development of individual digital skills
«Does the app also utilize and develop your digital skills?» (3 answers, not very informative)
T
h
e
i
The highest number of responses came to the question about innovative qualities of the app
Students find working with the app a interesting and effective. They have never been
using a mobile app for learning before.
«What makes the app innovative?» (13 answers)

31. Multiple-choice questions
Summary
Accorning to students opinions
the most supportive element to learning
are instructional videoes
Majority of students (61,8%) have made
instructional video themselves.
Less than 10% (8,8%) think they will not
get involved in producing videoes themselves

32. Students are actively involved in constructing the content of the learning resource .
Majority of students consider MILAGE App intresting, easy to use, innovative, helpful,
and supportive to individual and collaborative work.
(Majority of Likert scale questions (86%) received answer «Yes» as the most frequent one.)
Working with the app was referred to as a helpful, convenient and a different way of learning
( Response to the question wbout innovative qualities of the app as a learning resource)
All in all
Likert scale questions:
Open-ended questions:
Multiple choice questions:

33. Takk for meg