Designing a questionnaire, pre-coded or closed, open-ended or open, postal/mail or online self-completion

1. Questionnaires
The nature and use of questionnaires
Some principles and problems
Types of questionnaire
The validity of questionnaire research

2. The nature of use of questionnaires
• Respondents supply answers
• Filling in responses (self-completion questionnaire) or
giving information to an interviewer (face-to-face or
telephone)
• Cheap, fast and efficient
• Main tool of measurement in positivist sociology
• Easily quantified, can be analysed more ‘scientifically’
than qualitative data
• Positivism – approach in sociology that believes society
can be studied using similar scientific techniques to
those in the natural sciences, such as physics,
chemistry and biology

3. Pilot survey
• Must obtain comparable data
• Questionnaires can’t be changed once survey has
begun
• Pilot studies are used to test questions, make
sure their meaning is clear, and to ensure layout
and wording are suitable for the intended sample
• Important to clear up problems ad avoid wasting
time and money on poorly designed
questionnaires

4. Designing a questionnaire
• As short as possible, with clear layout and
instructions
• Clear and neutral language, and the avoidance
of ‘leading questions’
• Confidentiality

5. As short as possible, with clear layout
and instructions
• Clearly laid out and well printed
• Easy to follow and fill in
• Minimum questions required – respondents may
be unwilling to answer long lists of questions, or
may stop giving serious thought to their answers
• Simplest questions first and complicated
questions at the end – not put off right away
• Enough alternative answers (including ‘don’t
know) for correct expression of views

6. Clear and neutral language, and the
avoidance of ‘leading questions’
• Questions should be capable of being answered ‘yes’, ‘no’ or ‘don’t know’
• Neutral terms so respondents don’t feel like they need to give a particular
answer. Leading questions do this, and are likely to produce
distorted/invalid results e.g. ‘Why do you think promiscuous sex is wrong?’
encourages people to accept that is wrong
• Simple, everyday language so people understand
• Time options rather than ‘Do you watch TV a lot?’
• Only questions they are likely to be able to answer accurately should be
asked (expected to know about, able to remember)
• Questions should mean the same thing to all respondents
• Offensive questions should be avoided

7. Confidentiality
• Those being surveyed should be reassured
that their answers will be kept confidential or
anonymous

8. Types of questionnaires
• Pre-coded or closed
• Open-ended or open
• Postal/mail or online self-completion

9. Pre-coded or closed questionnaires
• Structured
• Multiple-choice
• Pre-set closed questions
• Limited number of multiple-choice answers

10. Advantages and disadvantages of pre-
coded or closed questionnaires
Advantages Disadvantages
• Quick to complete
• Produces standardized data;
easily classified and produced in
quantitative statistical form
• Reliable (likely to produce
similar results if carried out by
another researcher)
• Data collected can produce new
theories or test existing ones
• Enables comparisons between
different groups and
populations – answering the
same questions
• Meaning of questions may not
be clear to some respondents
• Extra questions cannot be asked
or added for respondent's to
fully explain their answers
• Imposition problem – risk that
the researcher, when asking
questions, might be imposing
their own views or framework
on the people being
researched, rather than getting
at what they really think

11. Open-ended or open questionnaires
• Unstructured
• Pre-set questions though no pre-set answers
• Individuals write their own answers or dictate
them to the interviewer

12. Advantages and disadvantages to
open-ended or open questionnaires
Advantages Disadvantages
• More valid date –
respondent able to express
their true view rather than
having to pick from what
the researcher considers
important views.
Imposition problem is less
• More detail and depth to
the answers
• Difficult to classify and
quantify the results –
answers may be unclear to
researcher, researcher may
be biased when
interpreting meaning of
answer
• Wide variety of answers 
difficult to compare results
with other similar research

13. Advantages and disadvantages of
postal/mail or online self-completion
questionnaires
Advantages Disadvantages
• Cheap compared to interviews
• Data can be collected from a wide
geographical area for cheaper cost
• Results quickly obtained – most that
get returned are returned within a
couple of weeks
• People can reply at leisure and not just
when an interviewer is present – more
precise answers
• Questions on personal, controversial,
embarrassing or sensitive issues (e.g.
depression, stigmatized diseases like
HIV and AIDS) can get better result
because interviewer is not present
• No interviewer bias – answers being
influenced by presence of interviewer
• Non-response (50% response is
considered good)
• Unrepresentative sample – only
people interested or educated on the
topic may take part
• Questions may be interpreted
differently – no interviewer present to
explain questions
• No way of knowing the right person
completed the questionnaire

14. The validity of questionnaire research
Imposition
• Imposition problem
• Researchers have already
decided what the most
important matters are
• Impossible for respondents to
express feelings and subtle
shades of opinion in statistical
form
• Impossible to measure
subjective factors such as
nature and strength of religious
belief
Validity
• No guarantee that people will
tell the truth
• Different meanings attached to
the wording of questions e.g.
people would rather consider
themselves ‘middle class’ than
‘lower class’ but if given the
option of ‘working class’ that is
what they’d pick
• Researchers need to be aware
of how such meanings may
differ between classes, ethnic
groups, age groups, etc.