2. THINGS YOU SHOULD KNOW ABOUT…
news and calendars to keep students engaged in the learning
community. Also focusing on the Apple platform, Seton Hill
University announced plans to offer an iPad to every full-time student in fall 2010, a technology chosen both for its mobility and the
promise of easy future access to e-textbooks. A joint outreach
program undertaken by Carnegie Mellon University and the
University of California, Berkeley, called Mobile and Immersive
Learning for Literacy in Emerging Economies (MILLEE) seeks to
support a group of English teachers in rural India with m-learning
applications designed for grade-school students. The children access these activities via their mobile phones to work on English
skills in the classroom as well as in the fields on days when they
help with farm work.
Why is it significant?
As learning management systems adapt to the mobile platform, m-learning may become a common tool for exploration by
tech-savvy faculty. The use of mobile devices seems a natural fit
for distributed learning and field activities in that handheld technology can not only accompany the learner almost anywhere but
also provide a platform that is rapidly evolving and always connected to data sources. Learning management systems may drive
campuses to recognize the potential of this always-on, anyplace
technology that lowers the physical boundaries to learning and
extends the classroom. Ease of use offered by mobile devices supports lifelong learning, and because the devices themselves are
integrated into everyday life, they facilitate authentic learning.
Ultimately, it might be the ubiquity of these student-owned devices that ensures their use as teaching and learning tools. The
rising popularity of smartphones should promote the development of cloud-based applications that work on multiple devices.
While some m-learning applications may be provided by colleges
and universities, mobile technology in the main provides an inexpensive layer of functionality to the institution, capitalizing on an
infrastructure that is increasingly supported by cloud services and
by the technology that students bring to campus.
What are the downsides?
Hardware for mobile learning represents a wide range of
platforms, screen sizes, and functionality, and no clear standards
exist for development that address all of the tools available. As
a result, colleges and universities can find infrastructure issues
tricky to resolve. The cost of smartphones and data plans is out
of reach for some students, and adoption and ownership is uneven. While the screen size on many mobile devices enforces
simplicity of design, the small screens and keys are difficult for
some to use effectively, and the additional strain on battery life
imposed by mobile apps can be frustrating. Because m-learning
is an emerging market, there remains a dearth of applications
designed specifically for learning, and repurposing existing lesson materials for the mobile platform might add to faculty workload. The eclectic mix of devices and mobile formats, which are
generally subject to student and faculty choice, could delay
m-learning development, and standards may be slow to emerge
in an environment where manufacturers are often trying to decide
whether to merge their mobile devices with slates, tablets, or
e-readers. Finally, while the devices can go anywhere with students, they might not engage students for long periods of time,
as mobile learning activities are subject to frequent interruptions.
Where is it going?
New kinds of devices are emerging, blurring the distinctions
between phones, PDAs, e-readers, and other types of hardware.
Future mobile technologies will be able to present textbooks, create data visualizations, aid library research, and foster contextual
Regardless of the directions taken by mobile manufacturers, newer
incarnations of these devices are sure to provide easier access and
better support for multimedia creation and collaborative applications. Field learning from art appreciation to zoology may soon
find support from mobile devices pulled from a student pocket and
used on the spot to check data, snap a photo, record location data,
make a blog entry, or enter a question on the class discussion
board. This rapid access to data, available wherever and whenever
questions arise, could change our learning landscape, altering the
way we solve problems.
What are the implications for teaching and
The cell phone is currently the most common platform for mlearning, lending itself to collaborative and project-based efforts
that leverage its potential to support the communication requirements of a team. Where wireless networks are available, or where
smartphones with data plans have access to cell networks, mobile
lessons and exercises can leverage the ability to gather information from a variety of interdisciplinary sources in a wide array of
formats while exploiting the value of location-based learning. In
developing countries where mobile devices are available at a fraction of the cost of other computing hardware, m-learning has extended the infrastructure of distance education to outlying areas
that have previously been poorly served. Regardless of the hardware employed, as demand requires that more applications be
re-authored for mobile formats, institutions may find it necessary
to overhaul data-sharing and content-delivery techniques to support the mobile platform.
EDUCAUSE is a nonprofit membership association created to support
those who lead, manage, and use information technology to benefit
higher education. A comprehensive range of resources and activities
is available to all EDUCAUSE members. The association’s strategic
directions include focus in four areas: Teaching and Learning; Managing
the Enterprise; E-Research and E-Scholarship; and the Evolving Role of
IT and Leadership. For more information, visit educause.edu.
4. Figure 1. Student Ownership of Technology Devices
A majority of undergraduates
own about a dozen devices
eReader Games Computer WebHandheld Desktop
Essential technology: Percentage of
students who said these technologies are
“extremely valuable” for academic success
of students have
for academics in
the past year
Count the ways: How smartphone owners
use their devices for academics
Texting other students about coursework
Looking up information on the Internet in class 45%
The data in this infographic comes from the ECAR National Study of Undergraduate Students and Information Technology,
2011. The full study highlights additional student perspectives captured in an online survey administered in 2011.
context. An example here could be contextaware applications because they draw on
surrounding environments to enhance the
Col l a b o rat ive : a c t ivit ie s p romoti n g
learning through social or other interaction. Collaborative learning is based
on the role of social interactions in the
process of learning, and mobile devices
offer new opportunities to engage in
and lifelong: activities that
support learning outside a dedicated
learning environment and formal curriculum. This category includes activities that
are embedded in everyday life, thus emphasizing the value of mobile technologies in
Learning and teaching support: activities
that assist in the coordination of learners
and resources for learning activities. This
category includes activities or applications
that are not instructional in nature but that
support instruction and thus learning. For
example, access to the learning management system, schedules, library resources,
or student support services.
Using these categories can help us further
clarify existing current mobile learning efforts
and also suggest opportunities for future work.
For instance, it is not uncommon for many
institutions to begin their mobility work in the
learning and teaching support category (for
example, providing mobile access to course
schedules), sometimes outside an instructional
context. But doing so builds tolerance and
understanding of mobile applications with our
students, a tolerance that can be leveraged for
learning in the classroom.
Another way to organize institutional mobile
learning activities is by the degree to which they
are integrated into the curriculum. I propose
Level 1: Service-related mobile content (e.g.,
access to the schedule or course offerings,
library resources and services, campus tram
5. Level 2: Generic mobile instructional appli-
and guidelines for the components of a mobile
experience to consider before developing or
launching a pilot or evolving an existing project.
Case 1: Identifying Focus Areas
Because mobile teaching and learning is still an
emergent area, many studies are exploratory
in nature. In other words, they don’t establish
any specific criteria for measurement or identify areas for examination. They simply insert
technology and applications into the learning
environment and observe how instructors and
learners are using them. While it’s fine and even
desirable to have an exploratory component in
a pilot, it’s important to identify areas of focus
or outcomes that you want to report on when
the study is finished.
In their study on assessing mobile learning in
a large blended classroom, Wang et al. identified several areas for analysis:3
cations (e.g., use of student response
systems, Twitter, or the learning management system)
3: Discipline-specific, customized
mobile learning (e.g., mobile applications
or tools that are developed to support a
particular set of learning objectives within
As noted earlier, many institutions
begin development of mobile content and
resources in areas that support students in
a non-instructional way, as described in level
1. Others support learning through the use
of mobile applications that are openly available at little or no cost to the user (level 2).
Some examples of this include backchannel
tools like Twitter or Poll Everywhere. Another
example might be enabling students to access
information (grade book) or content (assignments, readings) in the mobile-enabled
learning management system. These uses
are distinguished by the fact that they can be
used by any discipline and may be applied in
a variety of ways. Level 3 describes the use of
mobile applications and devices to support
specific discipline-based learning objectives.
In other words, this level supports the development of pedagogies that specifically make
use of the affordances of mobile learning.
For instance, Carleton College developed a
mobile digital flashcard program to support
the educational goal of helping language
learners expand their French vocabulary—
an area of persistent need.2 Level 3 activity
is more costly but often yields the highest
returns in the areas of improved learning,
increased student engagement, and additional time on task. Institutions do not
typically begin their mobile work here, but
when they do this kind of work, they collaborate with instructors, instructional designers,
and technology developers to create customized mobile software and applications.
Organizing mobile activities and efforts
according to the levels or categories described
above can support planning and strategy by
understanding where current mobility work
is taking place and by helping institutions
set goals for where they want to go in future
phases of mobility. The development and use
of frameworks to measure activity at each level
is, of course, critical in moving forward and
informing those directions.
Frameworks for Measuring Impact
In this section, I draw from four studies
or cases of institutional work in the area of
mobile learning. Each case offers some ideas
Student enjoyment and learning
Student interaction with other students
Student interaction with instructors
Impression of the mobile learning environment
Effect on study habits
The researchers designed survey questions
(see the study for additional information on the
surveys) organized around measures of activity,
efficiency, outcomes, and organization. They
also set out to measure student satisfaction,
level of interaction (student-student, studentinstructor), and sustainability of student
participation in mobile learning activities.
After collecting and analyzing the quantitative and qualitative feedback, they established
seven reliable evaluative dimensions for the
study of mobility’s effects on learners: overall
satisfaction, course organization, course activities, student interaction, instructor interaction,
relationship to content, and sustainability. Each
of these is described in detail in the study,
which provides an excellent framework for a
Case 2: Challenges and Goals
Identifying areas for study is only the first step in
evaluating mobile learning. Vavoula and Sharple
suggest several challenges and opportunities
and further contribute to our framework:4
Capturing learning contexts and learning
across contexts: Learning that happens in a
variety of spaces, inside and outside a classroom, is more challenging to capture than
that which is situated in a fixed, physical
learning environment. “This study suggests
that mobile learning research attempt to
6. capture the location of learning and the
layout of the space (where); the social
setting (who, with whom, from whom); the
learning objectives and outcomes (why and
what); the learning method(s) and activities
(how); and the learning tools (how).”
Identifying learning gains: Many mobile
learning studies set out to “improve student
learning” but struggle to isolate the effect
of mobility on learning and to define gains
in learning. In other words, how will you
know students learned more because of
a mobile application, and how will you
measure how much more they learned as
a result? This study suggests we be very
specific about what we’re looking for and
identify a few behaviors—for example,
where learners show responsibility for
and initiate their own learning (e.g., by
writing), are actively involved in learning
(e.g., spending additional time on a task),
or make links and transfer ideas and skills
(e.g., by comparing information).
Tracking behavior: Although rare, some
studies “monitor” student activity on mobile
devices with the use of tracking software.
This can be unreliable and present ethical
issues. An alternate approach is to ask
students to contribute to a daily or weekly
journal about their mobile learning experience: what they did, what went well, what
was challenging, and so forth. Doing so can
help to assess the dimensions established in
case 1 above.
The technology itself: Mobile technology
today spans a variety of devices, from
the small flip phone to tablets and even
netbooks. Studies should include some
component related to the usability of the
devices themselves and also to the extent to
which they are integrated into the learners’
technology constellation. In other words,
how well do the devices and the applications used work with other technologies
used on a daily basis? Is the experience
seamless, cumbersome, or incompatible?
How could it be improved?
The big picture: Mobile learning has the
potential to expand the learning environment well beyond the walls of the institution.
Vavoula and Sharples suggest the use of
Price and Oliver’s impact studies: anticipatory, ongoing, and achieved. “Anticipatory
studies relate to pre-intervention intentions,
opinions, and attitudes; ongoing studies
focus on analyzing processes of integration; and achieved studies are summative
studies of technology no longer novel.” This
phased approach is valuable as we plan the
future phases of our mobile studies. While
today’s studies may explore integration
with existing technologies and basic use,
tomorrow’s studies may look at applications
tailored to learner environments and needs.
As the technology evolves and matures, so
should our studies.
Formal and informal learning: Mobile tech-
nology is blurring the lines between formal
and informal learning because it enables
students to access and interact with
their learning environment and networks
anywhere, anytime. Students will have to
help us understand these contexts as they
continue to evolve into highly personalized
environments. As these come into focus,
we’ll better understand how and where
support is needed.
Case 3: Mobile Learning and Instructional Design
Traxler and Kukulska-Hulme, based on their
extensive review of mobile learning initiatives, suggest several characteristics of a good
mobile learning experience.5 In this list, we
could supplant mobile technology with any
technology, which demonstrates these are
universal instructional design principles. As
such, these could be used to develop a rubric
to measure our local mobile learning work. It
should also be noted that this list is particularly relevant when a technology is new or
emergent and may be less so as it evolves and
becomes increasingly ubiquitous.
Proportionate: What is the proportion of
time spent learning and applying the technology with the actual learning experience
or learning benefit? A cost/benefit analysis
is suggested, which will, to some extent,
depend on the tools (device and applications) you use and the extent to which
students are familiar with them.
Fit: How appropriate is the mobile tech-
nology to the learners, to the learning, and
to the course delivery mode? Is there a good
fit between the technology and the desired
outcome, between the technology and the
tools students already have or know, and
between the technology and the style of
the instructor? Is the technology well integrated into the students’ personal learning
Alignment: To what extent are the learning
goals mapped to the technology’s affordances? It might be helpful to make these
connections for students, especially if the
technology or its application is experimental.
Unintended results: Since mobile learning
is still relatively new, it will be useful to find
7. a way to identify unexpected outcomes—
a qualitative approach will likely be most
effective. Doing so will help inform future
phases of mobile initiatives.
Consistency: How consistent is the tech-
This case offers several key questions for our
What previously unavailable digital (on
mobile devices) content and resources will
we make available to students?
nology’s application (and results) across
learners? Is the use or application reliable?
How consistent is it across devices and
How will students be introduced or oriented
These five components can be integrated
into a mobile learning framework and into the
very design of the initiative. They can be fashioned into a rubric to be used as we design,
evaluate, and evolve our pilots.
will it be organized? In other words, will
the materials be held in a single repository,
or will there be a portal to access content
from a variety of sources? As the volume of
content increases, we need to be mindful of
how the learning environment is structured
and how that structure enables learning.
If the environment is cumbersome and
complex, students are not likely to use it.
Case 4: The Student Experience
In a case study of a medical school’s mobile
learning experience, Nestel et al. highlight
several findings that are worth considering as
we build our mobile initiatives.6 The institution in the study offered a graduate medical
program where students largely spent their
first year on campus, while the remaining
three years were spent almost entirely in clinical settings. All students were issued a mobile
device at the start of the program and worked
inside a virtual learning environment that made
use of two main teaching and learning tools,
Blackboard and Interlearn, for course management and facilitation of interaction online,
along with various other licensed software.
There were 57 students in the entering cohort
for the case study, and through student questionnaires and interviews, the study sought
to address two questions: In what ways does
mobility support learning, and what areas
This institution sought to enhance students’
learning experience through the use of
mobile technology to provide access to the
Internet, software, and information repositories and also to enable information sharing
within and between cohorts. In this case,
students had access to more than 10 different
sources (e.g., lecture notes, problem-based
learning materials, external weblinks, summative assessments, and so forth). These goals
are not uncommon in mobile initiatives, and
many institutions will provide students with
a significant amount of digital content to
support their learning.
Students in the study offered a few suggestions for improvement. They asked for more
robust resources and for more electronic learning
resources (especially audiovisual resources), along
with more flexibility in accessing the materials.
Since there was so much content, they felt they
would have benefitted from guidelines and support
on managing their virtual learning environments.
to the content and the environment within
which it resides?
Where will this content reside, and how
How will students manage their resources?
How are students able to interact and
engage with the content? Are they able to
annotate, download, share, or highlight the
content? How long are they able to access
the materials, from how many devices, and
in what ways?
Some of these issues can and should be
addressed together with students. Students
can help us create or understand their learning
environments and the features they need to
make the best use of them. As mobile technologies enable the proliferation of content and
resources, effective information management
Implications for Planning
Earlier in this brief I noted the distinction between the components we wish to
measure and the way in which we go about
measuring them. Once we have developed the
components of the mobile experience or the
framework, we can then explore approaches for
measurement. For instance, we might determine that the best way to identify unintended
results is through focus groups, live observations, and journal analysis. Alternatively,
you might measure the impact of a particular
mobile technology on student comprehension of a particular concept through the use
of pre-tests and post-tests or a comparison
group. The idea is that once you have a framework established, a variety of measurement
approaches and methodologies are available,
but the framework must come first.
Each of the cases above contributes some
advice for planning:
Capture and analyze learning in the context
in which it occurs.
Assess the usability of the technology and
8. how it affects students’ personal learning
Look beyond measurable cognitive gains into
changes in the learning process and practice.
Consider organizational issues in the adop-
tion of mobile learning practice and its
integration with existing practices.
Span the life cycle of the mobile learning
innovation that is evaluated, from conception to full deployment and beyond.
And finally, when working with emerging
technologies, it’s important to remember to
evolve our studies and frameworks as the
technology evolves. These approaches should
not be static, but should change as the adoption of the technology and corresponding
pedagogies mature. Constructing our research
so we are informing future phases is always
important, but it is especially important when
working with an emergent technology.
1 Laura Naismith, Mike Sharples, and Jeffrey Ting,
“Evaluation of CAERUS: A Context-Aware Mobile
Guide,” 2005, http://www.mlearn.org.za/CD/papers/
2 “Closing in on Vocabulary Acquisition: Testing the
Use of iPods and Flashcard Software to Eliminate
Performance Gaps,” SEI Case Study, EDUCAUSE
Learning Initiative, March 2012, http://net.educause
3 Minjuan Wang, Daniel Novak, and Ruimin
Shen, “Assessing the Effectiveness of Mobile
Learning in Large Hybrid/Blended Classrooms,”
h t t p : //w w w. d o c s t o c . c o m /d o c s / 3 8 9 6 4 5 6 3 /
4 Giasemi N. Vavoula and Mike Sharples, “Challenges
in Evaluating Mobile Learning,” Proceedings of
mLearn 2008, available from http://www.mlearn.org/
5 John Traxler and Agnes Kukulska-Hulme, “Evaluating
Mobile Learning: Reflections on Current Practice,”
6 Debra Nestel, Andre Ng, Katherine Gray, Robyn Hill,
Elmer Villanueva, George Kotsanas, Andrew Oaten,
and Chris Browne, “Evaluation of Mobile Learning:
Students’ Experiences in a New Rural-Based Medical
School,” BMC Medical Education 10, no. 57, 2010,
Read this QR Code to do the next activity, please.
The EDUCAUSE Learning Initiative (ELI) is where teaching and learning professionals come to learn, lead, collaborate,
and share in the context of an international forum. Members benefit from the expansive emerging technology research
and development that takes place collaboratively across institutions. To learn more about ELI, visit educause.edu/eli.