Survey Research 101

Online Surveys 101
A practical how-to

What is survey research?
Asking structured questions
Online surveys
Telephone surveys
Structured face-to-face interviews

Surveys versus other qualitative methods

Is a survey the right research method?
Yes
No
I know a lot about the topic
I need to summarize the findings in numerical format
I don’t need a lot of detail and nuance
I need to generalize the results to the population
I need to predict the likelihood of a certain thing happening
I want to measure incremental change
I don’t really know much about the topic
I don’t need to summarize the findings in numerical format
I need to communicate a lot of detail and nuance
I’m interested in case study information
I’m not planning on predicting the likelihood of anything
I don’t need to measure any change

Is it time to do a survey?
Clear idea of all possible questions and answers?
Formulated all questions and possible answers?
Yes
Do a survey
Yes
No
Pilot test the questions
Yes
No
Discovered questions and answers through
Qual research?
Do more Qual research
No

Asking good questions
Mutually exclusive and exhaustive
No overlap between answers
All possible answers
No double-barreled questions
Wrong: “What do you think about our response times and customer service?”
Right: “What is your opinion of our response time?”
Focus on opinions or beliefs of participants
Wrong: “How usable is our Web site?”
Right: “What are users’ opinions of the site’s look and feel?”
Close-ended is best
Cuts down on analysis time
Demographic questions at the end

Sampling and its implications
Probability Sample
Non-probability Sample
You can predict the likelihood of any single person being selected to participate
Has some randomized element to it
Allows for you to compare your findings to what you would expect with pure random events
Allows for prediction of accuracy
Allows for generalizing to the population at large
Can use inferential techniques
You cannot predict the likelihood of any person participating
Has no random element to it
Cannot be used to predict future likelihood of anything
Cannot be used to generalize to the population at large
Still provides some insight, particularly if you ask everyone in a group the same question
Does not allow to estimate the accuracy of the results
Cannot use inferential techniques

The lay of the land: descriptive statistics
Summarizes the responses
What is the most “typical” response?
Mean
Mode
Median
How much variation is there between responses?
Standard deviation
Inter-quartile range

Predicting behavior: inferential statistics
Infers or predicts future behavior
Which respondents are more likely to…
Cross-tabulation
Which factors most influence
Behavior
Satisfaction
etc

The tools you need
Online Survey Tool
Statistical Analysis Software

The skills you need
Knowledge of the topic
Ability to write good questions
Familiarity with the survey tool
Some basic knowledge of statistics
A framework for writing the report
Important to write the outline of the final report before you start!

The payoff you’ll get
“More men are dissatisfied with the checkout process.”
“Younger users are much more unhappy with customer service that older users.”
“Higher-income individuals are the most unlikely to recommend this site to friends and family.”

What you won’t learn…
Why aren’t people coming to our site?
How can we innovate on our current design?
Why don’t people finish the checkout process?
What are the main characteristics of the people who don’t come to our site?

Copernicus ConsultingDesign research and strategy
http://www.copernicusconsulting.net