Would you answer these questions?

Would you answer
these questions?
Susan Frede
VP, Research Methods & Best Practices
March 2016

Copyright © 2016 Lightspeed, LLC. Proprietary and Confidential. All rights reserved.
Poor quality survey design
contributes to…
• Low completion rates
• High dropout rates
• Speeding
• Straight-lining and other suspicious behavior
• Panel attrition
• Higher sample costs
Potentially leading to bad data,
unrepresentive samples, and
ultimately incorrect business decisions

Mobile may finally
force better
survey design

Let’s look at blinded screen shots
from real-world surveys
Think like a respondent…
• Would you answer/be able to answer each one?
• What would be your reaction to each one?
• What is bad about each one?
• How would you improve each one?

• Unnecessary scene setting/introduction
• Most respondents are not going to read
all of this

Company XYZ

• Need clear instructions – format for date
• Some respondents are hesitant to give exact
birth date for privacy/security reasons
• Better to ask age or age range question

• Use appropriate terminology for specific
country
• Current account is a common UK term for
checking account
• It is important on global projects to get
regional input

• Need to clearly define terminology
• What is meant by charitable donations? Money?
Used clothing/household items? Time?
• Five years is a long time period for respondents
to recall

• Poor layout
• Check boxes need to be much closer to
the answer text
• Could also shade every other row
• Or when a row is selected it is shaded

• Missing a ‘none’ option
• Could not proceed without choosing one
of the four answers

• Extremely long list of almost 100 TV programs
• Best bet is to cut the list down or randomly
select a subset
• Show in multiple columns; however, old
research has shown mixed results with
multiple columns

• Another long list with nearly 100 items
• Have to respond with something for every item so this is
extremely time consuming for the respondent
• Some would split this into multiple questions; however, this
just causes respondents to complain about repetition and
increases dropout rates with each additional grid
• Again, best bet is to cut the list down or randomly select a
subset of movies
• Might also try a more interactive question style
• Watch wasted space…large gray box and company logo

• Factual questions should be in
alphabetical order so respondents can
easily find their response

• Horizontal scrolling is a huge no-no…there are five more
columns when scrolling right
• Unequal column widths can cause bias in the answers
• Would be better if first ask what social networks
respondent has used in last seven days
• Then specifically ask what activities the respondent has
done on each site
• This will likely allow social networks to run across the top
with no scrolling and activities down the side
• Could also use a more interactive question style

• Column headings should always be visible
• Scrolling up and down leads to errors

• This is a large grid…could never read this on a
mobile phone
• Two key improvements:
1. change to five point scale
2. use a more interactive question style such as
dynamic grids
• Other issue is topic relevancy…can someone who
has only heard of the brand really answer these questions?

www.lightspeedgmi.com
info@gmi-mr.com
CONTACT US

Would you answer these questions?

More Related Content

Similar to Would you answer these questions?

More from Kantar

Would you answer these questions?