MacCallum, K. & Parsons, D. (2017). Evaluating a Mobile Toolkit for Designing Mobile Learning Activities. Proceedings of 16th World Conference on Mobile and Contextual Learning. Cyprus. ACM.

1. Dr David Parsons | CITRENZ 2017
Evaluating a Mobile Toolkit for
Designing Mobile Learning
Activities
Kathryn MacCallum and David Parsons

2. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
● Development and testing of Mobile Learning Activity
Design Analyser
● First cycle of 6 stage design science process from
motivation through to evaluation
● Key features of the analysis tool
● Data gathered from use by a small group of educators
● Relationship between learning theory and mobile
learning activity design
Agenda

3. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
● Mobile Learning Activity Design Analyser
● Analyses a learning activity design by responding to a
set of statements about its features
● Output is a visualised and aggregated graphic
describing how the learning activity leverages key
aspects of specific learning theories
● Alongside this graphic there are some suggestions on
how the overall design of this activity might be
improved and developed further
The Tool

4. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
● Behaviourism
● Constructivism
● Experiential learning
● Situated cognition
● Communities of practice
● Connectivism
Six Learning Theories

5. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
The phase of work described here is the first cycle through
a 6 stage design science process
1. Problem identification and motivation
2. Define the objectives for a solution
3. Design and development
4. Demonstration
5. Evaluation
6. Communication
Peffers, K, Tuunanen, T, Rothenberger, M. & Chatterjee, S. (2007). A design science research methodology for information
systems research. Journal of management information systems, 24(3), 45-77.
Methodology

6. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
1. Problem + 2. Solution Objectives
1. Problem
Support is needed to integrate learning theories into
mobile learning activity design
2. Solution objectives
Create a simple software tool to provide some guidance
for how learning theories can be integrated into mobile
learning activities

7. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
3. Design and Development
Design requirements
● Targeted feedback to the user based on inputs
● Should run on mobile devices (modelling)
Development
● HTML5 + JavaScript
● JQuery UI
● Slider Pips
● ChartJS

8. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
4. Demonstration
● The app itself is partly covered
in our IAmLearn eBook chapter
● This presentation is about the
evaluation rather than the app
itself
● However we will run a quick
demonstration
tinyurl.com/manalyser

9. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
Participants
● 15 educators, experienced in delivering mobile learning
activities, were invited to participate voluntarily and
anonymously in the study
● 7 agreed to participate
● 5 were able to complete all four stages of the study

10. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
4 Part Process
Part 1: Participants were asked to outline a mobile
learning activity (targeted learners, learning objective(s),
description, tasks, mobile features, types of learning)
Part 2: Participants used the Mobile Learning Activity
Design Analyser tool
Part 3: Participants used the feedback from the tool to
modify their activity designs
Part 4: Participants used the tool again, this time to reflect
on their design modifications

11. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
5 Learning Design Activities
1. Does 'fitbit' monitoring increase user motivation?
Identify motivators in the FitBit phone app
1. Place-based learning
Critical reflections of place using video
1. Introduction using geo-tagged video
New distance students sharing videos
1. Maths problem solving via site trip
Travel to the zoo - logistics analysis
1. The returning soldier
ANZAC memorial sculpture trail

12. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
Integration of
learning theories
into the initial
designs (left) and
the final designs
(right)
5. Evaluation (4 complete data sets)

13. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
Perceived Usefulness

14. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
+ Feedback
● “Think it is a really useful tool.”
● "The real 'gain' I got from it was from the first cycle
where it suggested where I could increase certain
learning theory content/processes. This allowed me to
reflect on the 'feedback' score and ask myself 'did I
really want the activity to go this way?' So, by getting
the first feedback I could see areas where I genuinely
thought I could go back and add some more detail.”

15. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
- Feedback
● “I suppose it 'assumes' that an activity should have
equal portions of each learning theory? I am not sure I
agree with that...”
● "I almost needed to do this a third time."
● “need more support on actual mobile learning design.”
● “would need a bit more info about some of the
meanings of the terms.”

16. Dr David Parsons | CITRENZ 2017Kathryn MacCallum & David Parsons | mLearn 2017
Conclusion
● A way to more authentically engage in evaluating a
mobile activity
● Dynamic nature of the tool enables a more visual and
interactive engagement
● Makes the educator more aware of the design of the
application
● Enables them to critically evaluate and provide
suggestions to improve the design (based on
appropriate learning theory)