Chapter13

265 views

Published on

Research Methods in Education 6th Edition

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
265
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Chapter13

  1. 1. SURVEYS © LOUIS COHEN, LAWRENCE MANION & KEITH MORRISON
  2. 2. STRUCTURE OF THE CHAPTER • What is a survey? • Some preliminary considerations • Planning a survey • Low response and non-response, and how to reduce them • Survey sampling • Longitudinal, cross-sectional and trend studies • Strengths and weaknesses of longitudinal, cohort and cross-sectional studies • Postal, interview and telephone surveys • Internet-based surveys • Comparing methods of data collection in surveys
  3. 3. KEY FEATURES OF SURVEYS • Gather data on a one-shot basis; • Represent a wide target population • Generate numerical, often large scale data; • Provide descriptive, inferential and explanatory data; • Manipulate key factors and variables; • Gather standardized information; • Ascertain correlations; • Remove contextual data; • Gather data from multiple choice, closed questions, test scores or observation schedules; • Support or refute hypotheses about the target population; • They are piloted; • Make generalizations about, and observe patterns of response in, the targets of focus; • Gather data which can be processed statistically.
  4. 4. SAME SAMPLE / COHORT OVER TIME TYPES OF SURVEY LONGITUDINAL CROSS- SECTIONAL TREND/ PREDICTION STUDIES SNAPSHOTS OF DIFFERENT SAMPLES AT ONE OR MORE TIMES SELECTED FACTORS CONTINUOUSLY OVER TIME
  5. 5. PLANNING A SURVEY • Specify the exact purpose of the enquiry: – Primary topics – Subsidiary topics – Specific information requirements • Specify the population on which the survey is to focus; • Specify the resources that are available: – Human (e.g. for administering and processing survey) – Material – Financial – Administrative – Temporal – Geographical – Software
  6. 6. PLANNING A SURVEY • Data collection – Researcher-administered – Self-administered – Postal survey – Telephone survey – Internet survey – Face-to-face interviews – Email • Self-reporting – May lead to biased reporting
  7. 7. PLANNING A SURVEY • Problem definition • Sample selection • Design of measurements • Concern for participants
  8. 8. STAGES IN SURVEY DESIGN DEFINE OBJECTIVES DECIDE INFORMATION SAMPLING INSTRUMENTATION PILOT DATA COLLECTION DATA ANALYSIS REPORTING TRAINING
  9. 9. STAGES IN CONDUCTING A SURVEY 1. Define the objectives; 2. Decide the kind of survey required; 3. Formulate research questions or hypotheses; 4. Decide the issues on which to focus; 5. Decide the information that is needed to address the issues; 6. Decide the sampling required; 7. Decide the instrumentation and the metrics required; 8. Generate the data collection instruments; 9. Decide how the data will be collected; 10. Pilot the instruments and refine them; 11. Train the interviewers (if appropriate); 12. Collect the data 13. Analyze the data; 14. Report the results.
  10. 10. PROBLEMS IN SURVEYS • Poor sampling • Poor question design and wording (failure to operationalize) • Incorrect or biased responses • Low response or non-response
  11. 11. HOW TO IMPROVE RESPONSE RATES • Identify where the problem lies: – The survey never reached the intended people; – People refuse to answer ; – People may not be available; – People may not be able to answer the questions; – People may not actually have the information requested; – People may overlook some items in error; – The survey was completed and posted but failed to return. – The pressure of competing activities on the time of the respondent; – Potential embarrassment at their own ignorance if respondents feel unable to answer a question;
  12. 12. HOW TO IMPROVE RESPONSE RATES – Ignorance of the topic/no background in the topic; – Dislike of the contents or subject matter of the interview; – Fear of possible consequences of the survey to himself/herself or others; – Lack of clarity in the instructions; – Fear or dislike of being interviewed; – Sensitivity of the topic, or potentially insulting or threatening topic; – Betrayal of confidences; – Losing the return envelope or return address; – The wrong person may open the mail, and fail to pass it on to the most appropriate person.
  13. 13. HOW TO IMPROVE RESPONSE RATES • Ensure that the topic is interesting and motivating; • Ensure that it is easy to complete and layout is attractive; • Include a covering letter that explains the survey; • Choose a good time to conduct the survey; • Follow-ups and polite reminders; • Pre-paid return stamped addressed envelopes; • Institutional affiliation, survey sponsorship or support from a high status agent; • Financial incentives and rewards for return; • Making instructions about responses and return very clear; • Avoid open-ended questions unless these are really important; • Avoid placing open-ended questions at the start of a questionnaire;
  14. 14. HOW TO IMPROVE RESPONSE RATES • Have easy-to-follow instructions and spacing of the text; • Flatter the participants without being seen to flatter them; • Providing information about the research through a covering letter and/or advance notification; • Making the survey look very unlike junk mail; • Consider asking the respondents for an interview to complete the survey questionnaire; • Deliver the questionnaire personally rather than through mail; • Ensure that the questions or items are non- judgemental; • Assure confidentiality and anonymity.
  15. 15. HOW TO IMPROVE RESPONSE RATES • Follow a sequence: – Send non-respondents a friendly reminder after ten days, stressing the importance of the research; – Send a further friendly reminder ten days after the initial reminder, stressing the importance of the research; – Make a telephone call to the respondents shortly after the second reminder, indicating the importance of the research.
  16. 16. CONSIDERATIONS IN SAMPLING • A probability and non-probability sample; • Confidence level and interval; • The desire to generalize, and to whom; • The sampling frame; • The sample size; • The representativeness of the sample; • Access to the sample; • Anticipated response rate.
  17. 17. LONGITUDINAL DATA ARE AFFECTED BY . . . • History (events occurring may change the observations of a group under study); • Maturation (participants mature at different speeds and in different ways); • Testing (test sensitization may occur – participants learn from exposure to repeated testing/interviews); • The timing of cause and effect: some causes may produce virtually instantaneous effects and others may take a long time for the effects to show; • The direction of causality not always being clear or singular. • Comparability of data over time; • Attrition (dropout); • Respondents’ memory and recall capabilities.
  18. 18. MEMORY IS AFFECTED BY . . . • The time that has elapsed since the event took place; • The significance of the event for the participant; • The amount of information required for the study – the greater the amount, the harder it is to provide; • The contamination/interference effect of other memories of a similar event (i.e. the inability to separate similar events); • The emotional content or the social desirability of the content; • The psychological condition of the participant at interview. • Hindsight.
  19. 19. ADVANTAGES OF POSTAL SURVEYS • Reach many people • Comparatively cheap • Can be completed at respondents’ preferred time • No risk of interviewer bias • Can reach scattered populations • Can gather sensitive data (as nobody else present)
  20. 20. DISADVANTAGES OF POSTAL SURVEYS • Low response rate • No check on understanding • No check whether all items are completed • Need to have a very simple format • Presentation matters affect completion
  21. 21. ADVANTAGES OF INTERVIEW SURVEYS • Improves response rates • Can clarify queries from respondents • Can stimulate the respondent to give full answers • Can be flexible (e.g. in item sequence) • Benefit from non-verbal communication • Can build in trust and rapport • Ensures that only the respondent answers the questions
  22. 22. DISADVANTAGES OF INTERVIEW SURVEYS • Interviewer characteristics affect responses • Conduct of interview affects responses • Flexibility can reduce standardization • Costly in time, travel and training
  23. 23. ADVANTAGES OF TELEPHONE SURVEYS • Reduce bias in the researcher or the interviewee • Reduce costs of time and travel • Easy to find more people to contact
  24. 24. DISADVANTAGES OF TELEPHONE SURVEYS • Not everyone has a telephone • Not everyone is available for interview • People are ex-directory • The person answering the call may not be the most suitable person • Multiple choice and rating scale questions are difficult • Order effects can be strong • People may lie or hang up if questions are unwelcome, sensitive, too long, too many • Lack of non-verbal cues
  25. 25. ADVANTAGES OF INTERNET-BASED SURVEYS • Many people can be reached • Cheap to administer • Quick to process results • Can reach difficult populations • Anonymity and non-traceability • Novelty value • Respondents can complete the questionnaire from home; • Respondents can complete it at a time to suit themselves; • Respondents can complete the survey over time; • Complex skip-patterns can be created and organized by the computer, so that participants do not have to understand complicated instructions;
  26. 26. ADVANTAGES OF INTERNET-BASED SURVEYS • The software can prompt respondents to complete missed items or to correct errors; • Computer can check incomplete or inconsistent replies; • For each screen, the computer can provide an on- screen indication of how much of the questionnaire has been completed; • Reduction of researcher effects; • Human error is reduced in entering and processing data; • Additional features may make the survey attractive; • Greater generalizability may be obtained as Internet users come from a wide and diverse population; • Greater authenticity of responses may be obtained.
  27. 27. DISADVANTAGES OF INTERNET- BASED SURVEYS • Not everyone has internet access • Limited control over sampling • Ethics (e.g. traceability) • Hardware and software problems • Expertise of respondents • Visual presentation takes on added signficance • Limited number of lines per screen • People give a minimal response • Order effects (early responses affect later ones) • People stop if it is too long or complicated

×