Lachlan Stokoe from Vision Critical presented best practices for mobile research. He discussed how mobile research is evolving and highlighted key considerations like identifying compatible question types, being aware of potential biases, and keeping surveys short to optimize for small screens. Stokoe also emphasized the importance of testing surveys on multiple devices and deploying thoughtfully based on respondents' usage patterns to maximize completion rates. The presentation concluded with a recap of best practices like maximizing limited screen space, controlling scrolling, and targeting multimedia sparingly.

1. A
Presenta*on
from
The
Fes*val
of
NewMR
–
Training
Day
3
December
2012
All
copyright
owned
by
The
Future
Place
and
the
presenters
of
the
material
For
more
informa:on
about
NewMR
events
visit
NewMR.org
Sponsored
by:
See
the
eXhib:on
for
booths
from
media
partners
&
supporters
Best
Prac*ce
In
Mobile
Research
Lachlan
Stokoe,
Vision
Cri:cal

2. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
BEST PRACTICE IN MOBILE RESEARCH
FESTIVAL OF NEWMR TRAINING DAY
3RD DECEMBER 2012
LACHLAN STOKOE
VISION CRITICAL, AUSTRALIA

3. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
A BIT OF BACKGROUND...
Vision Critical has been Investigating
mobile research since 2008, both in
terms of technical design and research
on research
Launched mobile-web optimised
surveys during 2011
Conducted hundreds of multi-modal
and mobile only studies globally
My background is in quantitative
research

4. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
MANY
PEOPLE
LIVE
A
‘MOBILE-­‐CENTRIC’
LIFE...

5. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
Many operating systems
Unique browser
applications
Different input modalities
Screen size/resolution
Lack of Adobe Flash/
Javascript
Over
6,500
web-enabled
devices
MOBILE
WEB-­‐BASED
APP-­‐BASED
SMS/WAP-­‐BASED
BUT
IT’S
STILL
A
JUNGLE
OUT
THERE...

6. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
MOBILE
RESEARCH
IS
AN
EVOLVING
FIELD:
Some
examples
of
current
and
poten*al
future
applica*ons...
The
ability
to
seamlessly
connect
mul*ple
elements
of
these
together
in
an
ongoing
dialogue
with
your
respondents
(while
keeping
each
ac*vity
short
and
engaging)
QUANTITATIVE
In-­‐situ
research
&
geoloca:on
(capturing
consumers/
shoppers
in
the
moment)
Event
based
research
(at
the
event
or
as
soon
as
possible
aNerwards)
Post-­‐experience
sa:sfac:on
Diary
studies
Website
evalua:on
via
pop-­‐ups
QUALITATIVE
Mobile
ethnography
Photo/video
upload
and
share
Event
experience
Integra:on
with
social
media
(Facebook
app)

7. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
There are additional considerations when conducting mobile research
You should consider these in the research design process, NOT get to the end
of that process and tack on mobile as just an additional deployment option
Think of it is an opportunity to challenge the status quo and approach things
differently

8. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
IDENTIFY AND OPTIMISE
BE AWARE OF QUESTION TYPES & BIASES
KEEP THE LENGTH UNDER 10 MINUTES (LESS IF NOT INCENTIVISED)
MAXIMISE AVAILABLE SCREEN SPACE
CONTROL FOR SCROLLING
TARGET MULTIMEDIA FOR IMPACT
QA FOR ALL DEVICES
DEPLOY INTELLIGENTLY
BEST
PRACTICES
FOR
MOBILE…

9. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
IDENTIFY
AND
OPTIMISE…
Understand what questions work and how
they will display in different operating
systems and design questionnaire
accordingly
Question types that aren’t supported may
be skipped by the software
…FOR THE DEVICES SUPPORTED BY YOUR SOFTWARE

10. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
UNDERSTAND
YOUR
MARKET…
A bias may exist in terms of smartphone
owners generally and/or particular types/
brands
…AND BE AWARE OF POTENTIAL BIASES
In Australia, we found that there is an
inherent bias in Smartphone ownership,
but no difference between different
brands

11. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
KEEP IT SHORT… AND SIMPLE
KISS…
The same survey will take 50% longer on a
Smartphone compared to a desktop
Respondents are willing to give some leeway
(within reason) – up to 10 minutes is a fair
estimate for an incentivised survey (which is a
6-7 minute standard online survey)
Consider breaking longer surveys up into
chunks and running as a series of shorter
surveys – this obviously introduces other
challenges

12. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
REAL
ESTATE
IS
AT
A
PREMIUM…
Remove superfluous words from question
and response code text to keep length
short
Only use images in response buttons when
there are a small number of options or it is
absolutely necessary for question
comprehension
Keep branding elements simple - avoid using
banners and logos
…SO LOOK TO MAXIMISE YOUR AVAILABLE SCREEN SPACE

13. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
EXCESSIVE
SCROLLING
HURTS…
Where it makes logical sense, you may be able
group 3-4 short related questions on the one
screen e.g. age, gender, marital status
Keep answer categories short (try to keep to
6 or less, particularly when multiple response
or scale ratings)
If you have more than 5 items in a grid you should
consider splitting over multiple screens. NB:
Different software will display grids in different ways
so make sure you understand the layout/number
of screen implications for your survey
Minimise scrolling as much as possible. Where it is
unavoidable you should try to keep it to ‘one-way’.
…SO CONTROL FOR LIMITED AND LOGICAL SCROLLING

14. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
USE
MULTIMEDIA
SPARINGLY…
Keep in mind that data usage
limitations and screen load times can
be an issue when using multimedia –
introduces drop out risk and
respondent backlash
Excessive use of multimedia can
dilute its impact
…FOR MINIMUM PAIN AND MAXIMUM IMPACT

15. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
…QA ON AS MANY DIFFERENT DEVICES AS POSSIBLE
ALLOW
TIME
TO
‘CHECK
IT
AGAIN
SAM’…
If doing a multi-modal study make sure
you check both completion methods
thoroughly.
Just to reiterate – there are many operating
systems, browsers, screen sizes, and other
quirks to mobile devices. Know what ones
your software covers and test on as many
different platforms as you are able to.
If possible, also test on an un-supported
platform to see what this experience is like.
Can a respondent see a non-optimised
version of the survey or are they precluded
from taking part?

16. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
GIVE
PEOPLE
A
CHOICE
TO
RESPOND
WHEN,
WHERE
AND
HOW
THEY
PREFER…

17. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
…TO MAXIMISE CHANCE OF RAPID RESPONSE
Be aware of the sample implications of making a study
mobile only – even when advised it is mobile only many
will try to complete on a desktop and drop out when that
fails
Be clear in your communication to respondent about
the choice or requirement of completing via mobile
The response window for mobile is tighter than for PC –
Need to deploy at times of the day people are most
likely to be able to complete straight away on their
mobile – morning/evening commutes/lunchtime/after
8pm (likely to vary by target group)
Special additional challenges/considerations
apply to weekend deployments
DEPLOY
INTELLIGENTLY…

18. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
IDENTIFY AND OPTIMISE
BE AWARE OF QUESTION TYPES & BIASES
KEEP THE LENGTH UNDER 10 MINUTES( LESS IF NOT INCENTIVISED)
MAXIMISE SCREEN SPACE
CONTROL FOR SCROLLING
TARGET MULTIMEDIA FOR IMPACT
QA FOR ALL DEVICES
DEPLOY INTELLIGENTLY
RECAP:
BEST
PRACTICES
FOR
MOBILE…

19. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1
IDENTIFY AND OPTIMISE
BE AWARE OF QUESTION TYPES & BIASES
KEEP THE LENGTH UNDER 10 MINUTES( LESS IF NOT INCENTIVISED)
MAXIMISE SCREEN SPACE
CONTROL FOR SCROLLING
TARGET MULTIMEDIA FOR IMPACT
QA FOR BOTH MOBILE AND DESKTOP
DEPLOY INTELLIGENTLY
AND
REMEMBER…
DESIGN FOR MOBILE
DON’T JUST TACK IT
ON

20. Lachlan Stokoe, Vision Critical, Australia
Festival of NewMR 2012 – Training Day – Session 1