Learn The Best Practices of Survey Design by designing the survey you want to meet your market research needs.

1. Survey Design Webinar: Proven
Methodologies for Surveys That Work

2. Greg Timpany Anup Surendran
Vice President, Product & EngineeringSenior Research Director, Global Knowledge;
Partner with Anova Market Research

3. #SurveyDesignTips

4. What’s on the agenda today?
● How to Identify the Need
● Why do You Want to Run a Survey
● How to Design a Survey that Does What You Want
● Turning Goals Into Real Survey Questions
● Collecting Answers to Your Carefully Crafted Questions
● Turning Resources into Insightful and Actionable Data

5. All Is Not What You See
Photo by Uri Golman
The root cause of business
issues often lie below the
surface.
Management may see a
decline in sales, but it is up
to the researcher to isolate
the cause.

6. Identifying the Need
● Research does not thrive if it is kept in a
vacuum
● Look below the surface
● Query clients in order to understand
what is known

7. It All Starts With a Plan
Source: Research Rockstar

8. Do we really need a survey?
Secondary Research
Pros
● Can be less costly in terms of
time and budget
● Leverages the work of other
analysts
Cons
● It was designed to meet
someone else’s research needs
and may not directly address
your areas of interest
Primary Research
Pros
● Directly designed to answer
your research questions
● Provides you maximum
flexibility for analysis
Cons
● Cost - primary projects
typically cost more than
secondary research, in terms
of time and budget

9. Why do you want to run a survey?
If secondary research including: online search,
published reports, or an examination of existing
in-house data will not allow you to answer your
client’s questions then…
It is time for a survey!

10. Hey! What about qualitative research?QuantitativeQualitative

11. Designing a survey that works!
● Know your intended audience
● Know the overarching research questions
that need to be answered
● Balance the informational needs
● Respect the respondent’s time and
willingness to share their opinions

12. Survey Flow
Surveys have distinct blocks designed to
guide the respondent through the process.
For example:
Respondents who do not meet screening
criteria will be exited out.
Non-purchasers may be sent down one
branch, while purchasers sent down another.
Judicious use of survey logic ensures that questions are
appropriate for the type of respondent you are speaking to.

13. Survey Blocks
● Introduction:
○ Survey duration - give respondent’s a realistic time estimate
○ Remind them their opinions are secure
● Screeners:
○ Typical screeners include demographics (age, gender, region), level of
responsibility or purchase ability (B2B), product/service usage, or
brand/company awareness
○ Screeners are useful for establishing quotas
● Body
● Demographics
● Final Thoughts
● Redirects

14. Question Types - Single Select
Multiple Choice (Single Select):
Asks respondents to select a single category. This is
a go to question type for survey designers.
Keynotes:
● Keep category lists as brief as possible,
preferably 5 - 9 items.
● Randomize to prevent order bias, unless your
list is alphabetical, e.g. states, IT certifications,
etc. or ranged, e.g. age and income.
● Can use drop down lists.
● Include “Other (please specify)” where
appropriate.

15. Question Types - Multiple Response
Multiple Choice (Select all that apply):
Asks respondents to select all options that apply from a
category list.
Keynotes:
● Keep category lists as brief as possible, preferably
5 - 9 items.
● Randomize to prevent order bias.
● Remind respondents they can select all that apply.
● Make sure to include categories for “None” and
“Other (please specify)”
● Serves a good input question for other types such
as single select, rank order or constant sum.
● Each category is its own question. The sum of the
number selected can serve as a segmentation.

16. Question Types - Matrix or Grid
Matrix or Grid:
Useful for Likert-style questions as well as ratings. Items for consideration are in the rows and rating scale in
the columns.
Keynotes:
● Keep item lists as brief as possible, preferably 5 - 9 items. If you are rating more items then consider
formatting the list into multiple grids.
● Randomize items (rows) to prevent order bias.
● A/B test scale direction low to high vs. high to low. In the US we tend to think from low to high, but
that may not be the case in international settings.
● Can be created side by side so both importance and satisfaction, for example, can be measured
concurrently.
● Scaled questions, such as a feature’s perceived importance to the purchase decision, can be subjective.
Consider rank order, constant sum or Max-Diff question types.

17. Question Types - Matrix or Grid

18. Question Types - Rank Order
Rank Order:
Requires that the respondent order their selections
based on some metric such as preference.
Keynotes:
● Keep items to be ranked as brief as possible,
preferably 5 - 9 items. Respondents may have
difficulty ranking more than this number of
items.
● Randomize items to prevent order bias.
● Source data from a prior question, such as a
multiple response, can be used to feed a rank
order.
● Is less subjective than scaled questions.
● Creates ordinal data.
● Can use drag and drop (not advised for mobile)

19. Question Types - Constant Sum
Constant Sum:
Requires that the respondent allocate a series of
points, typically adding up to 100, across items.
Keynotes:
● Keep list as brief as possible, preferably 5 - 9
items. Respondents may have difficulty
ranking more than this number of items.
● Randomize items to prevent order bias.
● Source data from a prior question, such as a
multiple response, can be used to feed a
constant sum.
● Works well when tied to a numeric question,
such as amount spent during last shopping
trip.
● Is less subjective than scaled questions.
● Creates metric data.

20. Question Types - Image Choice
Selecting images:
Requires the respondent to select or rate an image or
images.
Keynotes:
● Great for soliciting market feedback on corporate
images, such as logos, or for providing a higher
level of engagement using pictures that are
common knowledge, such as a bowl of
strawberries or a sports car.
● Ensure that your images are clear at various
resolutions and can be seen on different devices.

21. Question Types - Open-Ends
Capturing Text:
Requires respondents to share thoughts in their own words.
Keynotes:
● Comes in a few different flavors
● Where to place Open-Ends
● Should they be “required”
● Safe to use open-ended questions throughout the
survey, after you have gotten through screeners.

22. Question Types - Advanced Questions
Keynotes:
● Customer satisfaction can be measured
using the Net Promoter Score question.
● Pricing can be addressed with the Van
Westendorp price sensitivity model
(PSM).
● Discrete choice or max-diff use cases can
be leveraged with the conjoint models.
● Other options include the ability to
generate heat maps, useful for testing
visual imagery, and the opportunity for
respondents to rate auditory or video
content.

23. Formatting Considerations
● Lists vs. Dropdowns
● Branded vs. Blind
● Number of Questions per Page
● Fonts
● Use of Space
● Text

24. Formatting Considerations - Mobile
● Readability
○ use mid-length scales (3 - 5 points) and avoid the
need to pinch or zoom
● Clickability
○ It’s easier to click with a mouse than with your
thumb. Be generous when sizing response buttons
● Loadability
○ Keep your images smaller and use fewer of them.
You can use more questions per page to reduce
the number of pages to be loaded.

25. Traps to Avoid
● Managing Time
● Leading the Respondent
● Double-barreled Questions
● Insensitivity
● Overlapping Categories
● Thinking Small
● It Doesn’t Apply
● Failure to Follow-Up
● Death By Grid

26. Thank You
Contact Information
For more info, please contact us at
​Greg Timpany
greg@anovamarketresearch.com
919-673-0795
Anup Surendran
anup.surendran@questionpro.com
512-590-3369
@questionpro