1. A PEDAGOGIC ASSESSMENT OF MOBILE
LEARNING APPLICATIONS
Jorge Arús Hita, UCM, jarus@ucm.es
Cristina Calle Martínez, UCM, cristinacalle@ucm.es
Pilar Rodríguez Arancón, UNED, prodriguez@flog.uned.es
ATLAS (Artificial Intelligence Techniques for Linguistic Applications)
ICDE-UNED, 7-9 March 2013

2. Outline
• Background and motivation
– SO-CALL-ME
– Why mobile learning?
• Research goal
– App evaluation
• Methodology
– Rubric-based
– Criteria for rubric creation
• Results
– Technical aspects
– Types of apps
– Differentiating features
• Conclusion
– What results show
– What we can do now

3. Background and motivation
• Research carried out within the SO-CALL-ME project
(Social Ontology-based Cognitively Augmented
Language learning Mobile Environment; FFI 2011-
29829)
• Mobile Learning is receiving widespread attention, as
attested:
– EU initiatives to promote it, e.g.:
• Framework Programmes of Research and Development
• MOBIlearn (with the US)
• eMapps
• M-Learning (in UK)
– Increasing presence of mobile learning at generalist
academic conferences, e.g. ICDE 2013

4. Research goal
• Examination of both the qualities and limitations of the most
salient EFL MALL applications available at the moment by
assessing their features from a pedagogic point of view
– Pedagogic in the sense that we do not focus here on the technical
specifications of the apps, but rather on the kind of EFL teaching
or practising they provide
– Starting point from which to gain knowledge and insights into the
features that are effective and suitable for learners using MALL
– to develop applications which help to improve the communicative
competences of the students/users in an independent and
effective way, putting to good use the full potential that
audiovisual material can provide

5. Methodology
• Creation of two templates (shared through
Google Drive):
– list of evaluated apps and their URLs
– Rubric for the evaluation of EFL apps
• Evaluation of EFL apps

6. Methodology
1. list of evaluated apps and their URLs :

7. Methodology
2. App assessment rubric:

8. Criteria for rubric creation
• Simplicity
– for relatively fast assessment which would enable
covering a fairly large number of apps in a
reasonable amount of time
• Geared towards our project’s specific needs:
– Cognitive value
– Similarity with the pedagogic aims of SO-CALL-ME
– complementarity with the pedagogic aims of SO-
CALL-ME

9. App evaluation
• 67 apps evaluated
– Some of them downloaded (when free to download
and once downloaded the app ran well)
– Most of them based on the scrutiny of the
information available on the website describing the
app
• Each of the three evaluators assessed different
apps
– Pro: Larger number of apps assessed
– Con: Potentially less reliable assessments, but …

12. Results
• A high number of apps presented technical problems at the time of downloading
or starting them
– Complaints in forums
– Observed on downloaded apps
• More apps for Apple than for Android (or others)
• A few of them can also be directly run from the Internet on a conventional
computer
• Prices
– Mobile versions of traditional dictionaries, textbooks, vocabulary or grammar tests, etc, (as
high as around 30 euros)
– Apps downloadable for a small amount –usually around 1 euro, and rarely above three euros
(e.g. Cambridge’s English Monstruo)
– Apps with an initial free sample pack and the possibility to download further packs for a small
amount (again, around 1 euro; e.g. the British Counsil’s LearnEnglish Grammar.
– Completely free apps ; e.g. The wide choice of apps available from the British Counsil.
– Full-fledged English courses , e.g. Busuu or EF’s EnglishTown (the price depends on the needs
of the user and/or seasonal offers)

13. Results
• Types of apps
– Games, very often aimed at children, e.g. apps available from Cambridge
English Online.
– App versions of dictionaries, handbooks and textbooks, e.g. Cambridge’s EFL
methods, dictionaries, etc;
– Apps providing vocabulary, grammar and/or pronunciation practice, such as
My Word Book, Johnny Grammar’s Quiz Master, 60 Second Word Challenge or
Sounds Right
• Some go beyond mere drilling or quizzing: listening comprehension practice by means of
podcasts and the exploitation thereof, e.g. Listen-to-English and A Cup Of English; apps
allowing conversation practice, e.g. English Feed, even with other users, e.g. The
Language Campus;
– Adaptation of online courses such as Busuu and EF’s EnglishTown to mobile
devices.
– Apps exploiting the use of language in context (most closely related to the
interests and goals of SO-CALL-ME): through podcasts –e.g. Learn English,
Talking Business English– videos –e.g. Learn English Audio & Video,
Conversation English– films –e.g. English Attack– and cartoons –e.g. Big City
Small World.

14. Results
• Differentiating features
– Drag & drop (e.g. Learn English Grammar)
– Drawing with one’s finger (e.g. Premier Skills)
– Connectivity with social networks (e.g. Language
City, Learn English, 60 Second Word Challenge and
Tongue Mystery English)
– Student’s Avatar (Quiz up)

15. Conclusion
• The results give us an idea of the qualities and limitations of the apps
evaluated, as a first step in the development of other apps that may fill
some existing gaps.
• The quantitative scrutiny here presented has allowed us to ascertain
the limited scope of many of the existing products.
– Not something in itself negative; it shows the efforts by methodologists,
linguistic engineers and technological enterprises to develop technology
which assists students to learn anytime and anywhere.
– Yet, it is also a fact that they tend to provide a rather fragmented language
practice: some vocabulary here, some grammar there, etc..
• Some of the MALL apps evaluated, however, do provide more
contextualized practice.
– We will look at these in more detail
• to learn from their strengths
• to integrate a sound pedagogy that may result in the creation of apps which,
without being a mere mobile version of traditional online courses, will provide
quality teaching and practice