1. Constructing questions for
interviews & questionnaires
Theory and practice in social research
Shahbaz Mehmood

2. Preface
• These slides include one theoretical
framework for constructing questions and a
small set of good and bad practices for
constructing questions
• This is not only or complete set, but gives
hints about what should you think when
constructing questions.

3. Errors in gathering data through
survey procedures are
• Responders’ failure to understand questions an intended
• A lack of effort or interest on the part of responders
• Respondents’ unwillingness to admit certain attitudes or
behaviours
• The failure of respondents’ memory or comprehension
process in the stressed condition of the interview
• Interviewers failures of various kinds (e.g. the tendency to
change wording, failures in presentation procedures and
the adoption of faulty recording process

4. Examples that illustrate the inadequate of
many of the questions
• Factual questions sometimes elicit invalid answer: As simple as age
• Relationship between what respondents say and what they actually do
in not always very strong: E.g. racism: Questions: Do you let a black
person inside a bar? Do they really?
• Responders’ attitudes, beliefs, opinions, habits interests often seem to
be extraordinary unstable
• Small changes in wording sometimes produce major changes in the
distribution of responses
• Respondents commonly misinterpret questions
• Answer to earlier question can affect to later question
• Order of options changes the respondents: e.g. middle option
• Format of question: Open vs. close
• Respondents answer even if they don't know about the topic
• Cultural context affects

5. The principal assumptions that have defined the
general orientation adopted by survey researcher in
the past
• The researcher has clearly defined the topic about which information is
required
• Respondents have the information that researcher requires
• Respondents are able to access the required information under the conditions
of the research situation
• Respondents can understand every question as intended
• Respondents are willing (or at least, can be motivated) to give the information
• Responses are more valid if they know why that is asked
• Answers are more valid if they are not suggested
• The research situation does not affect to the results
• The process does not affect the respondent
• All responses are meaningfully comparable

6. The traditional survey model
Researcher/
Interviewer Respondent
A carefully standardised physical
stimulus (i.e. question)
A response (i.e. answer)
expressed in term of a
standardised format provided by
the researcher

7. The key issue: the comparability
of aswer
• The reseacher must be clear about the nature of the information
required and encode a request for this format
• The respondent must decode this request in the way the researcher
intends it to be decoded
• The respondent must encode an answer that contains the information
researcher has requested
• The reseacher must decode an answer as the respondent intendedd it to
be encoded

8. Symbolic interactioinist theory
• Human beings interpret and define each other's actions. Not only
simple stimulus-response
• Human beings can be objects of their own attention. Argue
themselves, take pride themselves…
• Conscious social behavior is intentional behavior. Situation affects.
• Interpreting, planning and acting are ongoing process which begin
anew at every stage of a social process
• In situation there is a role and person wants to hold on it.
• Always exists
• In brief: social situation is constantly negotiating a shared definition of
the situation; taking one another's viewpoint into account; and
interpreting another's behavior

9. A model of symbolic interactionist view of question
answer behaviour
Encodes question, taking into
account own purposes and
presumptions/knowledge about
the respondent, and perceptions
of the respondent's
presumptions/knowledge about
self
Decodes question, taking into
account own purposes and
presumptions/knowledge about
the interviewer, and perceptions
of the interviewer's
presumptions/knowledge about
self
Encodes question, taking into
account own purposes and
presumptions/knowledge about
the interviewer, and perceptions
of the interviewer's
presumptions/knowledge about
self
Encodes question, taking into
account own purposes and
presumptions/knowledge about
the respondent, and perceptions
of the respondent's
presumptions/knowledge about
self
Interviewer Responder

10. The implication of the symbolic interaction
theory for social research
• Roles
• Respondents try to find a mutually shared
definition of the situation
• Interpretation of researcher acts
• Clues of what kind of information
researcher wants

11. Other
• Multidimensionality. Nothing is one-
dimensional
• Level of generality in the statements
• Level of generality in the topic
• Utterance frame: descriptive, explanatory or
evaluative

12. Defining topic properly
Encodes question.
The need to clearly specify the
information that is required. The
assumption that the respondent
have the required information.
The assumption that the
respondents can access the
information
Decodes question
Encodes answerDecodes answer
Interviewer Responder

13. Researcher has clearly defined
the required information
• Defined topic
– Global vs. local (29)
– Dimensions (economic, technical...)
• Is PL good thing: Marketing vs. Technical
– E.g. better social secure in cost of taxes
• Researcher has clear idea what kind of
information is required

14. Responder have required
information
• Hypothetical question => hypothetical Answer
• Willingness to answer vs. information to answer
• Opinions vs. facts
– Grounded opinions
– Serious opinions
• Responds are capable of verbalising information.
– Not hurry, give time to answer

15. Formulating intelligible requests
for information
Encodes question.
Formulating an intelligible
request for information.
Defining key terms for
respondents and virtues of
simplicity and breity
Decodes question
Encodes answerDecodes answer
Interviewer Responder

16. The meaning of individual words
• Context specific meanings
– Ambiguities words
– Different meanings
– Cultural decencies
• Relative difficulty of words
– foreign words
• The operation of unintended nuances associated
with apparently similar words
– Forbid vs. allow

17. The meaning of concepts
• Lack of empirical references
– How should be
– Difficul, unclear, wide concepts
• Context decencies (often, usually etc.)
• Concrete: Not: often, quite often, seldom.
Use 1,2,3,4,5

18. Structural complexities
• Number of words
• Less is better, but not complex ones
• Grammatical
– Asking too much in one question
– divide & conquer
– not what you like this and that!
• Not too much negatives: Double is bad

19. Good practices
• Descriptive introduction: I'd like to
describe…
• Explain before question
– The addition of clauses, phrases and
instructions
• Anecdotes
• Education of the respondent affects

20. Contextual influences on respondents'
interpretation of questions
Encodes question.
Encodes answerDecodes answer
Interviewer Responder
Decodes question
Contextual clues that influence
respondents’ interpretations

21. Clues afforded by either the
Question itself or its components
• Clues associated with the question
– Some existing idea always
– Leading questions
• Clues in components
– Additional phrases
– Note: Vegetables e.g. spinach not associated
vegetarian in general as intended

22. Clues in response options.
• Approve or disapprove - not only approve in question
• Scale
• Pre-set response options as memory clues
– May not remember other options
– Information range covered by the options.
– Who counts own averages?
– Abstract: no knowledge of normative levels
• Response biased with number of options
– Respondent favor first when reads, last when hears - vary the order
– Let responder tell, if no-one match

23. Impact of preceding questions
• Influence associated with prior Q. Issues:
– Does one have any knowledge?
– How deep is knowledge or how thorough
– How one interprets
• Psychological need for consistent
– Need to stay in one opinion
• Even-handedness
– Fair for all sides

24. Impact of preceding questions
(cont’d)
• The impact of the previous answers
• The impact of the overall situation
– Semantics of human mind.
– Willingness to give information

25. Contextual influences on respondents'
interpretation of questions
Encodes question.
Encodes answer
The kind of answer given is defined by the
dimension of the response framework that
is employed
Decodes answer
Interviewer Responder
Decodes question

26. Descriptive accounts
• How to describe the situation
• Perspectives

27. Level of social generality of responses
• You? Singular or plural?

28. Explanations
• Respondent can frame an explanation in
many different ways
• Why did you do X
– Causal antecedent - what caused to do X
– Goal antecedent - purpose for X
– Enablement factors - how possible
– Causal consequences - what happened after X
– The researcher's expectations

29. Evaluations
• Always relative
• Standards necessary does not exists
• Evaluative standard that are external to the
question
– Pleasant/easy flight - pilot vs. passenger
• Evaluative standard that are built in to the
question
– Agree -disagree

30. Limitations of human memory

31. Limitations of human memory
• Long term memory problems
– Nobody remembers all
– Ask from present to past, not past to present.
• Short term memory problems
– Simple questions

32. Filters

33. Filters
• Establishing the relevance of questions to
respondent
– Respondents tend to answer all the questions
• “I don't know enough”
– Don't know vs. don't have opinion
– Position of filters
• Middle and don’t know are problematic
• Middle category at all
– subjective viewpoint to strongly (dis)agree
• How strongly?

34. Reducing question threat

35. Reducing question threat
• Always some threat
• Biased
• Refuse

36. How?
• Casual, Do you happen to have…?
• Imputation of deviance, You know everyone does.
• Anonymous
• Lessen psychological immediacy of the Q (e.g. other instead self.
Numeric coding of alternatives)
• Decrease specificity of the information called (e.g. broad response
categories)
• Adopt knowing so that respondents have to confirm rather than
volunteer
• 'Kinsey' straight at eyes and ask
• Adopt indirect so that respond give answer without knowing
• Place threatening Q at the end of series

37. How? (cont’d)
• Door in the face: Ask direct, if does not
answer ask indirect/about e.g. salary.
• Ask long, don’t hesitate top repeat, give
time and encourage to use time
• The definition of Q threat

38. Threat causes bias
• Topics that are desirable and over-reported
– Be good citizen
– Be well informed and cultural person
– Have fulfilled of moral and social responsibilities
• Topics that are socially undesirable and under-reported
– Illness and disabilities
– Illegal and contra-normative behavior
– Financial status
• Feeling of guilt or personal fears
• Threat associated with the nature of the relationship between
interviewer and respond
– Social equity
– Fear of political or economical sanctions

39. Open vs. closed question debate

40. Open vs. closed question debate
• Coding responses to open questions
• Formulating response option for closed
question

41. Evaluation of the use of open questions
• Open Q doesn’t suggest answer -- or does it?
– Respondent’s use of "probes"
• Indicate level of knowledge
– Assumptions
• will answer open Q if knows
• don't try to answer open Q if don't know
• wil answer closed question if don't know
• Answers indicate the salience of the topic in the responder's mind
– Is there evidence?
• Avoid format effect
– Not in the middle!
– Is it in the closed?

42. Evaluation of the use of open questions
• Allows complex motivation
– Indicates more how respondent has interpreted it
– Indicate motivation that have influenced respondent's orientation to the
topic
– Indicates the frame of reference
• Problems associated with probing inadequate answer
– Turns to close
• Problems associated with coding response!

43. Evaluation assumptions
associated with closed questions
• Answers the question in the same way
-meaningfully comparable?
• Easier to answer
• More easily analyzed
– Interpretations
• Problems associated with recording responses to
closed questions
– Start with general instructions
– Explain why to answer

44. Measuring attitudes

45. Measuring attitudes
• List of respond alternatives p. 153
• Define topic clearly
• Applicability of the topic to respondents has to be
established
– Don’t know
• Respondents has to know what sort of answers they should
know
– Level of generality
• Specs of standards
– "Strongly agree"

46. Measuring attitudes
• Stimulus centered effects
– Number of categories
• 7 +/- 2
– Anchoring effects of he category labels
• The word in the positive/negative end?
• Problems in the batteries of rating scales
– Meaning can alter
– Some category
• long list. Don't know last when answering the first
• Ambiguity

47. Checks to ensure that questions
work as intended

48. Checks to ensure that questions
work as intended
• Editing rules (list p. 184)
• Piloting Question
– Observation hard, but must be done in the beginning to
ensure that questions work e.g. had to repeat?
• Question testing
– Rephrase the question in responder's own words
– Double interview
• Come in the beginning
– Allow aloud thinking

49. Esim KKK
• Kuinka monta parametroitua komponenttia on
tyypillisessä toimitettavassa tuotteessa (0=ei käytetä)?
• Miten paljon parametreja on tyypillisessä
parametroidussa komponentissa?
• Parametrien määrän vaihteluväli?
• Millaisia parametreja komponenteissa on?
• Millaisia arvoalueita parametreilla on (kokonaisluku,
arvoalue, yksi kokonaisluku joukosta , yksi annetuista
vaihtoehdoista, reaalilukuja, joukkoja)? Mitä muita?
Koponentti parametrinen vs
komponetin mukanaolo parametrista
Tieto vai mielipide
Halu vastata vai
vastaus olemassa?
Mikä on komponentti?
Mikä on tyypillinen
tuote?
Strukturaalinen kompleksisuus, liian
monimutkainen lause:
Kun toimitatte tuotteen, tuotteessa om
komponentteja. Kompoenenteilla voi
olla toimituskohtaisia parametreja.
Onko olemassa tällaisia
komponentteja, joissa on parametreja?
Montako tälläistä kompoenttia, jossa
on parametreja, tuotteessa on?
Vihjaa että pitäisi olla?
Jos ymmärsi aluksi väärin, halu
konsistenttiin ilmaisuun