This document discusses questionnaires as a research method. It defines questionnaires and notes they can be used to collect information about preferences, thoughts or behaviors systematically. Questionnaires have both strengths and weaknesses as a research method. They allow data collection from many individuals remotely but also have weaknesses like non-response bias, sampling bias, and social desirability bias. The document provides guidance on developing high-quality questionnaires, including avoiding problematic question types or wording, considerations for question order and sequence, and tips for interviewers when administering questionnaires. It also discusses sources of bias that can affect the validity of questionnaire responses.

1. Research 101: Questionnaires in Research
Harold Gamero

2. Questionnaires in Research
• Questionnaires collect information about preferences, thoughts or behaviors in a
systematic way.
• It can be used in descriptive, exploratory or explanatory studies.
• It is more suitable for research whose units of analysis are people.
• In the case of companies or groups, it is vital to identify the appropriate informant or
proxy.

3. Questionnaires in Research
• Can measure unobservable factors
• Data can be collected remotely & from
numerous individuals
• Non-intrusive
• Able to pick up very small effects (with
large samples)
• Economic in terms of time, money and
effort.
• Non-response bias.
• Sampling bias.
• Social desirability bias.
• Recall bias.
• Common method bias.
Strengths Weaknesses

4. Questionnaires
Definition
Instruments of data collection that contain a set of standardized questions or items.
• Questionnaires can be:
vs. Unstructured
• Respondents answer in their own words to
open-ended questions
• Usually to collect qualitative data
Structured
• Respondents choose their answers from
already defined alternatives
• They may contain or become measurement
scales or indices

5. Questionnaires
Some ways to apply these are:
• It is a type of self-administered questionnaire.
• One advantage is that the data are collected in computerized form.
• But it has lower response rates and higher risk of bias.
• The interviewer reads the questionnaire to the
respondent.
• Minimizes missing or erroneous data.
• Non-intrusive since the respondent fills out only
the form.
• It should be clear enough for the respondent to
understand on his/her own.
Self-administered questionnaires Assisted questionnaires
On-line questionnaires

6. Response forms
Two alternative answers.
Dichotomous
More than two alternatives without order or hierarchy.
Nominal
More than two alternatives with order or hierarchy.
Ordinals
Likert, semantic differential or Guttman scales.
Interval
Numerical answers with a real zero value.
Reason

7. Response forms
Quantitative, continuous or, metrics
Qualitative, categorical or non-metrical
Dichotomous
Nominal
Ordinals
Interval
Reason

8. Avoid:
Content & Wording
Unclear or complicated questions
• Simple language, active voice, without jargon or complex terms.
• All questions should have similar wording.
Negatively worded questions
• Best to avoid questions that include negations.
• Never use double negative questions.
Ambiguous questions
• Avoid questions that could be interpreted differently by different people.
• Do not use relative terms such as: a little, a lot, a little, just enough.
Biased or value-laden terms
• Avoid words or tones that motivate to answer in a certain way.
• Should governments pay for the rehabilitation of drug addicts?

9. Content & Wording
Avoid:
Questions with two or more sub-questions
• The questions must refer to a single specific issue.
• Does the teacher show respect for students' opinions and clarify their doubts?
Very general questions
• Especially in the measurement of constructs that by nature are
multidimensional.
• Are you satisfied with the service you received?
Very specific questions
• Unnecessary details may invalidate the question or questionnaire.
• What is the age of your children?

10. Content & Wording
Avoid:
Presumptive questions
• Avoid questions that contain a value judgment.
• What do you think are the benefits of the new IGV exemption law?
Imaginary questions
• Imaginary questions lead to imaginary answers.
• How much would you spend on…?
Questions whose answer are unknown
• Sometimes respondent don’t have enough information to answer.
• If they answer, the answer is inaccurate or completely wrong.
• Is your university in partnership with other foreign institutions?

11. Sequence of questions
• Start with easy and "superficial" questions (demographic or firmographic).
• Never start with open-ended questions.
• If it is a topic with a certain chronology, adapt the questions to follow the same order.
• Group questions by topic and do not move to another section until all questions are
completed.
• When moving to another section, mention it explicitly.
• Use filter or contingency questions if necessary.
(if you answered "Yes", skip to question 15).

12. Other “Golden” Rules
• Keep the survey as short as possible.
• Respondents usually do not give more than 10-15 min.
• Ensures confidentiality of responses.
• Mention how and for what purpose you will use the information.
• For organizations, it offers a copy of the final results of the study.
• He appreciates the participation of the respondents.
• Always conduct pilot tests.

13. Questionaires for Interviews
• Usually, to collect qualitative data.
• Interviews are more personalized than surveys but follow the same protocol.
• Because the interviewer is considered part of the instrument, some training is usually
required.
• Unlike surveys, the interviewer can clarify or elaborate on the respondent's answers.
• They take longer and require more resources.
• They offer rich data.

14. Surveys with interviews
Face-to-face interviews
• It may be suitable for some respondents, while others may feel uncomfortable.
• The interviewer must have the necessary skills to elicit the required
responses.
Focus group
• Usually for exploratory research.
• Interviewer should keep the dialogue on topic and avoid dominant
personalities.
On-line interviews
• It is typically based on random selection of individuals.
• Computer programs can allow the inmidiate transription.

15. Tips for the Interviewer
Preparing for the interview
• Data collection is highly dependent on the quality of the interviewer.
• The interviewer must be well acquainted with the research being conducted.
Locate interviewees
• It is necessary to know who will be interviewed.
• The interviewer must consider respondents' times and schedules.
Motivate respondents' response
• The interviewer should demonstrate enthusiasm and attention to the
interviewee.
• The importance of the study should be communicated.

16. Tips for the interviewer
Clarify doubts
• The interviewer must "put him/herself in the shoes" of the interviewee.
• If necessary, you should ask additional questions to ensure the quality of the
answers.
Evaluate the quality of responses
•The interviewer is in the best position to rate the quality of the answers.
•In addition to the written record of responses, the investigator should record
"key" phrases, gestures, tones or other elements of body language.
•Interviews are usually recorded on voice recordings, with prior authorization
from the respondent

17. During the interview:
Follow the order of the survey.
Do not end the interviewee's sentences.
Remain silently for the interviewee to continue or elaborate on an
answer.
Do not show approval or disapproval.
You may refer to other answers or ask for a more detailed
explanation.
When required, paraphrase the respondent's answer to confirm
understanding.
1
2
3
4
5
6

18. Non-response Bias
• Surveys usually have low response rates (15% - 20% for on-line surveys).
• This becomes a problem when the non-response is due to a systematic reason.
• To avoid this, the following measures can be taken:
➢ Ensure that the content is relevant.
➢ Make the questionnaire as user-friendly as possible.
➢ Advisable to have the approval or endorsement of a company or area.
➢ Follow up on unanswered questionnaires.
➢ Properly train the interviewer.
➢ Provide incentives that do not bias responses.
➢ Ensure data confidentiality and privacy

19. Sampling Bias
• This occurs when the sample selected is not representative of the population of
interest or when the method favours certain segments of the population.
• On-line surveys tend to include a higher proportion of students and young
people.
• Telephone surveys often exclude people who do not have landline telephones or
who are at work most of the day.
• e.g. Asking parents about their children's academic performance.

20. Social desirability bias
• It occurs when the respondent gives incorrect answers to "look good" or avoid
negative perceptions by other people, either to himself or to people close to him.
e.g. Do you consider your work team to be dysfunctional?
e.g. Have you ever downloaded music illegally?
• Although there is no precise way to eliminate this bias, it is possible to reduce its
occurrence by revising the questions in the questionnaire or modifying the
method of data collection.

21. Recall bias
• Occurs when the question asked requires the respondent to recall past events.
• Especially when asked about emotions, motivations or thoughts of that specific
moment.
• To minimize this bias, it is better to "anchor" the respondent's memory on a
specific event rather than perceptions about it.

22. Common method bias
• This is the risk of a possible spurious correlation between two variables, since
they are measured at the same time and with the same instrument.
• There are some statistical tests that can be used to measure the presence of
this bias.
• To minimize this bias, it is recommended to measure the variables at different
times or using different research methods.

23. Thank you.
Harold Gamero