Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Adler clark 4e ppt 09


Published on

Adler clark 4e ppt 09

  1. 1. Questionnaires and Structured Interviews Chapter 9
  2. 2. Introduction <ul><li>Questionnaires and structured interviews are the two most widely used methods of data collection in social science research. </li></ul>
  3. 3. Introduction <ul><li>Questionnaire </li></ul><ul><ul><li>A data collection instrument with questions and statements that are designed to solicit information from respondents </li></ul></ul>
  4. 4. Introduction <ul><li>Structured interview </li></ul><ul><ul><li>A data collection method in which an interviewer reads a standardized list of questions to the respondent and records the respondent’s answers </li></ul></ul>
  5. 5. Introduction <ul><li>Survey </li></ul><ul><ul><li>A study in which the same data are collected from all members of the sample </li></ul></ul><ul><ul><ul><li>Usually in the form of answers to questions </li></ul></ul></ul><ul><ul><ul><li>Often use large probability samples and cross-sectional study design </li></ul></ul></ul>
  6. 6. Introduction <ul><li>Respondents </li></ul><ul><ul><li>The participant in a survey who completes a questionnaire or interview </li></ul></ul>
  7. 7. Introduction <ul><li>Self-report method </li></ul><ul><ul><li>Another name for questionnaires and interviews because respondents are most often asked to report their own characteristics, behaviors, and attitudes. </li></ul></ul>
  8. 8. The Uses of Questionnaires and Interviews <ul><li>Questions can be asked about a variety of topics </li></ul><ul><li>It is common to use the self-report method to gather information on attitudes, beliefs, values, goals, and expectations </li></ul><ul><li>Questions can also be used to gather a person’s level of knowledge on something or behavior </li></ul>
  9. 9. Introduction <ul><li>Stop and Think </li></ul><ul><ul><li>The surveys by Public Agenda and Pew focused on attitudes towards national energy policies and health care, respectively. </li></ul></ul><ul><ul><ul><li>Suggest a list of other topics that you think you could ask questions about using a questionnaire. </li></ul></ul></ul><ul><ul><ul><li>Would you be concerned about the accuracy of the answers on any of these topics? </li></ul></ul></ul>
  10. 10. Focal Research <ul><li>Environmentalism Among College Students: A Sociological Investigation by Brandon Lang and Christopher W. Podeschi </li></ul><ul><ul><li>Asked college students questions about environmental issues </li></ul></ul><ul><ul><li>Interested in whether the students’ attitudes were connected to some background characteristics and to behavior patterns </li></ul></ul><ul><ul><li>Descriptive and explanatory purposes </li></ul></ul>
  11. 11. Focal Research <ul><li>Ethics </li></ul><ul><ul><li>The researchers submitted a proposal to their university’s Institutional Review Board (IRB) where it was approved. </li></ul></ul><ul><ul><li>The respondents were informed about the study’s sponsor, were assured that the surveys were anonymous and that they could skip any of the questions. </li></ul></ul>
  12. 12. The Uses of Questionnaires and Interviews
  13. 13. The Uses of Questionnaires and Interviews <ul><li>Although surveys are widely used there are important concerns about their validity </li></ul><ul><li>When using self-report methods it is based on the implicit assumption that people have the information being asked and they will answer based on their core beliefs and values </li></ul>
  14. 14. The Uses of Questionnaires and Interviews <ul><li>Answers to questions may be compromised if the participants </li></ul><ul><ul><li>Do not understand </li></ul></ul><ul><ul><li>Do not take the time to adequately answer the question </li></ul></ul><ul><ul><li>Influenced by current mood, events, etc. </li></ul></ul><ul><ul><ul><li>For example, research has shown that people who are in more positive moods answer more favorably compared to those who are feeling more negative (Martin, Abend, Sedikes, & Green, 1997) </li></ul></ul></ul>
  15. 15. The Uses of Questionnaires and Interviews <ul><li>Measurement error </li></ul><ul><ul><li>Error that occurs when the measurement we obtain is not an accurate portrayal of what we tried to measure </li></ul></ul><ul><ul><li>The giving of inaccurate answers to the questions </li></ul></ul>
  16. 16. The Uses of Questionnaires and Interviews <ul><li>Measurement error may also be affected by </li></ul><ul><ul><li>question wording </li></ul></ul><ul><ul><li>respondent’s level of knowledge on the topic </li></ul></ul><ul><ul><li>whether or not the respondent has an opinion about the topic </li></ul></ul><ul><ul><li>respondent having an accurate memory </li></ul></ul><ul><ul><li>respondent wanting to present a good image </li></ul></ul>
  17. 17. Participant Involvement <ul><li>Response rate </li></ul><ul><ul><li>The percentage of the sample contacted that actually participates in a study </li></ul></ul>
  18. 18. Participant Involvement <ul><li>Response rate is affected by </li></ul><ul><ul><li>The number of people who cannot be reached for the study (noncontacts) </li></ul></ul><ul><ul><li>The number who choose not to participate (refusals) </li></ul></ul><ul><ul><li>The number who are incapable of performing the tasks required of them </li></ul></ul><ul><ul><ul><li>For example, due to illness or language barriers </li></ul></ul></ul>
  19. 19. Participant Involvement <ul><li>Nonresponse error </li></ul><ul><ul><li>Results from differences between nonresponders and responders to a survey </li></ul></ul>
  20. 20. Participant Involvement <ul><li>When participants and nonparticipants differ in social characteristics, opinions, attitudes, values, or behavior – then generalization to a larger population is much more difficult </li></ul><ul><li>Some researchers have questioned whether higher rates of refusal automatically means more bias </li></ul>
  21. 21. Participant Involvement <ul><li>Stop and Think </li></ul><ul><ul><li>Have you ever been asked to complete a questionnaire or an interview? </li></ul></ul><ul><ul><li>What factors did you consider when deciding whether or not to participate? </li></ul></ul><ul><ul><li>Think about what you would do if you were on the other side. What would you do to try and convince someone to participate in a survey? </li></ul></ul>
  22. 22. Participant Involvement <ul><li>Participation in studies can best be understood within a social exchange context. </li></ul><ul><ul><li>Social exchange – means a cost/benefit approach </li></ul></ul><ul><ul><li>Once potential sample members are contacted, they must decide about cooperation after thinking about the costs and benefits. </li></ul></ul>
  23. 23. Participant Involvement <ul><li>Respondents are asked to give up their time, engage in interactions controlled by the interviewer, think about issues or topics that might cause discomfort, and take the risk of being asked to reveal embarrassing information. </li></ul><ul><li>Potential participants might worry about privacy and lack of control over the information </li></ul>
  24. 24. Participant Involvement <ul><li>Conversely, participants might want to participate because of an interest in a topic, a desire to share their views, or knowledge that their information will be useful to science and society </li></ul>
  25. 25. Participant involvement <ul><li>Possible ways to increase participant rates </li></ul><ul><ul><li>Advance mailings </li></ul></ul><ul><ul><li>Incentives </li></ul></ul><ul><ul><ul><li>Giving small gifts </li></ul></ul></ul><ul><ul><ul><li>Token cash payments </li></ul></ul></ul><ul><ul><li>Focusing on the interesting aspects of participation </li></ul></ul><ul><ul><li>Re-contacting participants to encourage participation </li></ul></ul><ul><ul><li>Minimize the costs of participation </li></ul></ul><ul><ul><ul><li>For example, time and possible embarrassment </li></ul></ul></ul>
  26. 26. Self-Administered Questionnaires <ul><li>Self-administered questionnaires </li></ul><ul><ul><li>A questionnaire that the respondent completes by him or herself </li></ul></ul><ul><ul><ul><li>Can be written or web-based survey </li></ul></ul></ul>
  27. 27. Self-Administered Questionnaires <ul><li>Interview </li></ul><ul><ul><li>A data collection method in which respondents answer questions asked by an interviewer </li></ul></ul><ul><ul><ul><li>Can be done by phone or in person </li></ul></ul></ul>
  28. 28. Self-Administered Questionnaires <ul><li>Group-administered questionnaires </li></ul><ul><ul><li>Questions administered to respondents in a group setting </li></ul></ul>
  29. 29. Self-Administered Questionnaires <ul><li>Group-administered questionnaires </li></ul><ul><ul><li>Advantages </li></ul></ul><ul><ul><ul><li>Allows the researcher to explain the instructions and answer questions. </li></ul></ul></ul><ul><ul><ul><li>Provides the researcher some control over the setting in which the questionnaire is completed </li></ul></ul></ul><ul><ul><ul><li>Allows the respondents to participate anonymously </li></ul></ul></ul><ul><ul><ul><ul><li>Helps in getting honest answers to sensitive questions </li></ul></ul></ul></ul><ul><ul><ul><li>Usually results in a good response rate </li></ul></ul></ul><ul><ul><ul><li>Inexpensive </li></ul></ul></ul>
  30. 30. Self-Administered Questionnaires <ul><li>Group-administered questionnaires </li></ul><ul><ul><li>Disadvantages </li></ul></ul><ul><ul><ul><li>There might be no group setting for the population the researcher wants to study </li></ul></ul></ul><ul><ul><ul><li>A group setting might raise concerns over the violation of voluntary participation </li></ul></ul></ul><ul><ul><ul><li>Groups typically have limits for the amount of time they spend on a survey </li></ul></ul></ul>
  31. 31. Self-Administered Questionnaires <ul><li>Mailed questionnaires </li></ul><ul><ul><li>Questionnaires mailed to the respondent’s residence or workplace </li></ul></ul>
  32. 32. Self-Administered Questionnaires <ul><li>Mailed questionnaires </li></ul><ul><ul><li>Advantages </li></ul></ul><ul><ul><ul><li>Inexpensive </li></ul></ul></ul><ul><ul><ul><li>Reasonably effective </li></ul></ul></ul><ul><ul><ul><li>No time requirement, allows the participant to not feel rushed </li></ul></ul></ul><ul><ul><ul><li>Answering in private, creates fewer social pressures and expectations </li></ul></ul></ul>
  33. 33. Self-Administered Questionnaires <ul><li>A questionnaire can be individually administered or administered as an Internet questionnaire </li></ul><ul><li>An individually administered questionnaire is hand delivered to the respondent and picked up after completion </li></ul><ul><ul><li>Similar to using mailed questions, but the expense is higher and has a better response rate </li></ul></ul>
  34. 34. Self-Administered Questionnaires <ul><li>An internet questionnaire is sent by e-mail or posted on a website </li></ul>
  35. 35. Self-Administered Questionnaires <ul><li>Coverage error </li></ul><ul><ul><li>A sampling error that arises when the sampling frame is different from the intended population </li></ul></ul>
  36. 36. Self-Administered Questionnaires <ul><li>Cover letter </li></ul><ul><ul><li>The letter accompanying a questionnaire that explains the research and invites participation </li></ul></ul>
  37. 37. Interviews <ul><li>Structured interview </li></ul><ul><ul><li>Interviewer reads a standardized set of questions and the response option for closed-ended questions. </li></ul></ul><ul><li>The interview has some similarities to a conversation, except that the interviewer controls the topic, asks the questions, and does not share experiences or opinions. </li></ul>
  38. 38. Interviews <ul><li>The set of instructions to the interviewer, the list of questions, and the answer categories make up the interview schedule </li></ul><ul><li>The use of structured interviews allows for some flexibility in administration, clarification of questions, and the use of follow-up questions. </li></ul>
  39. 39. Interviews <ul><li>Interviews typically have good response rates </li></ul><ul><li>Interviews are more expensive than questionnaires, because interviewers have to be hired and trained. </li></ul><ul><li>Using an interviewer adds the potential of interviewer effect , or the changes in respondents’ behaviors or answers that result from some aspect of the interview situation </li></ul>
  40. 40. Interviews <ul><li>In-person interviews </li></ul><ul><ul><li>The dominant mode of data collection from 1940 to 1970. </li></ul></ul><ul><ul><li>An in-person interview is conducted face to face. </li></ul></ul><ul><ul><li>In-person interviews are a good choice for questions involving complex reports of behavior, for groups difficult to reach by phone, or for respondents who need to see material or to consult records as part of the data collection, or when the interview is long, and high response rates are essential. </li></ul></ul>
  41. 41. Interviews <ul><li>In-person interviews </li></ul><ul><ul><li>Rapport </li></ul></ul><ul><ul><ul><li>A sense of interpersonal harmony, connection, or compatibility between an interviewer and a respondent </li></ul></ul></ul>
  42. 42. Interviews <ul><li>Phone interviews </li></ul><ul><ul><li>The dominant mode of survey data collection since the 1970’s </li></ul></ul><ul><ul><li>Preferred because it can yield close to the same results as an in-person interview at half the cost </li></ul></ul><ul><ul><li>Good for people who feel too busy for face-to-face interviews </li></ul></ul><ul><ul><li>New technology allows makes it cost effective to do computer-assisted telephone interviews in which data are collected, stored, and transmitted during the interview </li></ul></ul>
  43. 43. Constructing Questions <ul><li>Researchers make a series of decisions based on assumptions about how respondents read or hear the questions that are asked. </li></ul>
  44. 44. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Open-ended questions </li></ul></ul><ul><ul><ul><li>Questions that allows respondents to answer in their own words </li></ul></ul></ul>
  45. 45. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Closed-ended questions </li></ul></ul><ul><ul><ul><li>Questions that include a list of predetermined answers </li></ul></ul></ul><ul><ul><ul><ul><li>Answer categories must be exhaustive and mutually exclusive </li></ul></ul></ul></ul>
  46. 46. Constructing Questions <ul><li>Types of Questions </li></ul><ul><ul><li>Deciding whether to use open- or closed-ended questions involves several issues. </li></ul></ul><ul><ul><ul><li>Answer choices can provide a context for the question, they can make the completion and coding of questionnaires and interviews easier. </li></ul></ul></ul><ul><ul><ul><li>Respondent might not find the response that best fits what they want to say, and answer categories can be interpreted differently by different respondents. </li></ul></ul></ul>
  47. 47. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Open-ended questions </li></ul></ul><ul><ul><ul><li>All open-ended responses must be categorized before the researcher does statistical analysis </li></ul></ul></ul><ul><ul><ul><li>A limited number of answer categories must first be created for each question, so the data may be coded </li></ul></ul></ul>
  48. 48. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Coding </li></ul></ul><ul><ul><ul><li>The process of assigning data to categories </li></ul></ul></ul><ul><ul><ul><ul><li>Coding is a time consuming and expensive task that can result in the loss of data richness. </li></ul></ul></ul></ul>
  49. 49. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Screening questions </li></ul></ul><ul><ul><ul><li>Question that asks for information before asking the question of interest </li></ul></ul></ul><ul><ul><ul><li>Example </li></ul></ul></ul><ul><ul><ul><ul><li>Have you applied to a graduate or professional program? Yes or No </li></ul></ul></ul></ul><ul><ul><ul><ul><li>If yes, please list the program(s) </li></ul></ul></ul></ul><ul><ul><ul><li>Example </li></ul></ul></ul><ul><ul><ul><ul><li>It is important to ask someone if they voted in November, before asking whom they voted for. </li></ul></ul></ul></ul>
  50. 50. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>A screening question is often followed with a contingency question </li></ul></ul><ul><ul><li>A contingency question </li></ul></ul><ul><ul><ul><li>A question that depends on the answer to the previous question. </li></ul></ul></ul><ul><ul><ul><ul><li>Example </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>If yes, go to question #___ </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><li>Example </li></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>Do you work for pay outside the home? If yes, how many hours per week are you employed? </li></ul></ul></ul></ul></ul>
  51. 51. Constructing Questions <ul><li>Types of questions </li></ul><ul><ul><li>Occasionally researchers use indirect questions, where the link between the information the information desired and the question is not obvious </li></ul></ul><ul><ul><li>This is done by creating scenarios or vignettes about people or situations that the researcher creates to use as part of the data collection method. </li></ul></ul>
  52. 52. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Keep the questionnaire or interview as short as possible </li></ul></ul><ul><ul><li>Only the necessary questions planned for data analysis should be asked </li></ul></ul><ul><li>A preliminary draft or pilot test should be used with a small sample </li></ul><ul><ul><li>A pilot test is a preliminary draft of a set of questions that is tested before the actual data collection. </li></ul></ul>
  53. 53. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Avoid loaded words – words that trigger an emotional response or strong association by their use </li></ul></ul><ul><ul><li>Avoid ambiguous words – words that can be interpreted in more than one way </li></ul></ul><ul><ul><li>Don’t use double negative questions – questions that can ask people to disagree with a negative question </li></ul></ul>
  54. 54. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Don’t use leading questions – questions that encourage the respondent to answer in a certain way, typically by indicating which is the “right” or “correct” answer </li></ul></ul><ul><ul><li>Avoid threatening questions, or questions that make respondents afraid or embarrassed to give an honest answer </li></ul></ul>
  55. 55. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Don’t use double-barreled or compound questions – questions that ask two or more questions in one </li></ul></ul><ul><ul><li>Ask question in the language of your respondents, using the idioms and language appropriate to the sample’s level of education, vocabulary of the region, etc. </li></ul></ul><ul><li>Review Box 9.9 </li></ul><ul><ul><li>Examples of poorly worded questions and the suggested revisions </li></ul></ul>
  56. 56. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Question order & context </li></ul></ul><ul><ul><ul><li>Responses to questions can be affected by the question order as earlier questions provide a context for later ones and people may try to be consistent in their answers to questions on the same topic. </li></ul></ul></ul>
  57. 57. Constructing Questions <ul><li>How to ask questions </li></ul><ul><ul><li>Question order & context </li></ul></ul><ul><li>Consider a logical order that makes participation easy </li></ul><ul><li>To encourage participation start with interesting, no threatening questions, and save questions about sensitive topics for the middle or end </li></ul><ul><li>Conclude with “cool down” to minimize discomfort of participant </li></ul>
  58. 58. Summary <ul><li>Considerations when choosing a data collection method </li></ul>
  59. 59. Quiz – Question 1 <ul><li>Open-ended questions allow for respondents to answer </li></ul><ul><ul><li>in their own words. </li></ul></ul><ul><ul><li>according to pre-determined categories. </li></ul></ul><ul><ul><li>as often as they wish. </li></ul></ul><ul><ul><li>skip contingency questions </li></ul></ul><ul><ul><li>none of the above </li></ul></ul>
  60. 60. Quiz – Question 2 <ul><li>If close-ended questions are used, it is important that the responses are </li></ul><ul><ul><li>mutually exclusive. </li></ul></ul><ul><ul><li>exhaustive. </li></ul></ul><ul><ul><li>created by respondents. </li></ul></ul><ul><ul><li>a and b </li></ul></ul><ul><ul><li>All of the above </li></ul></ul>
  61. 61. Quiz – Question 3 <ul><li>Which type of survey tends to have the highest response rate? </li></ul><ul><ul><li>Mailed questionnaire </li></ul></ul><ul><ul><li>Phone interview </li></ul></ul><ul><ul><li>In-person interview </li></ul></ul><ul><ul><li>All three tend to yield high response rates </li></ul></ul><ul><ul><li>None of these yield high response rates </li></ul></ul>