The document summarizes a study that investigated differences in demographics and assessment scores between job applicants who completed remotely delivered assessments on mobile devices versus non-mobile devices. The study found that while a small percentage (1.93%) completed the assessments on mobile devices, these applicants tended to be younger, female, African American or Hispanic. Scores on a personality measure were similar across device types but scores on a general mental ability test were substantially lower for those using mobile devices, likely due to increased cognitive demands and distractions of small screens. The implications are that applicants should avoid high-stakes assessments on mobile devices where scores may be lower, and organizations should exercise restraint in facilitating such assessments on mobile until technology reduces observed score differences.

1. The Use of Mobile Devices in
High-stakes Remotely Delivered
Assessments and Testing
Winfred Arthur, Jr (Department of Psychology, Texas A&M University), Dennis
Doverspike and Alison E. Carr (University of Akron, USA), Gonzalo J. Muñoz
(Universidad Adolfo Ibáñez, Santiago, Chile), and
Jason E. Taylor (PeopleAnswers, Dallas, TX)
International Journal of Selection and Assessment Volume 22 Number 2 June 2014

2. Background
• Job applicants now have the opportunity
to complete remotely delivered
assessments on mobile, handheld small
screen devices such as smartphones, and
personal digital assistants.

3. Purpose
• To investigate demographic and score
differences between job applicants who
completed a remotely delivered high-
stakes assessment on a mobile device and
those who completed it on a nonmobile
device.

4. Methodology
• Sample
– 3,575,207 job applicants who completed an unproctored
Internet-based assessment between January 2011 and
April 2012.
• The assessment battery consisted of a personality measure
and GMA test:
– Internet-based Five Factor Model
– unproctored Internet based GMA: 60 items (28 verbal
and 32 numeric) four-alternative multiple-choice items.

5. Methodology
• All participants completed the assessments, which
were unproctored, on either a mobile or a
nonmobile device.
• they logged on to the assessment Internet test site
whenever they wanted from any device of their
choice, and also from any location of their choice

6. Result
• The percentage of applicants completing the assessment
on a mobile device was small, 1.93%, but nevertheless
represented more than 69,000 people.
• Mobile devices were slightly more likely to be used by
women, African-Americans and Hispanics, and younger
applicants.
– Mean age mobile = 23,88 vs Mean age nonmobile = 26,43
• Scores on a personality measure were similar for mobile
and nonmobile devices.
.

7. Result
• Scores on a general mental ability test were
substantially lower for mobile devices:
– increased information processing and working memory
demands in taking a cognitively loaded assessment on a
small screen device
– work in more distracting environments when completing
assessments on a mobile device, and their scores
reflect lower levels of concentration and on-task focus.

8. Implication
• Test Taker
– job applicants should avoid taking high-stakes
assessments on mobile devices since certainly in
the context of cognitively loaded tests, their
scores are likely to be lower than would
otherwise be the case.

9. Implication
• Organizational
– Exercise some restraint in encouraging or facilitating the
completion of high-stakes assessments on mobile
devices.
– at least until technological advancements (such as the
use of apps) can demonstrably reduce the observed
differences in mobile and nonmobile scores on
cognitively loaded tests and measures