Download as PDF, PPTX
Uploaded byRey Junco
Survey Methods - OIISDP 2015
The document outlines the survey process, including selecting questions, methods of administration, and data analysis. It discusses survey types, measurement levels, and best practices for writing questions, such as clarity and response options. Additionally, it delves into statistical concepts like power, effect size, and sampling techniques to ensure accurate and reliable data collection.
More Related Content
![Research Design [Creswell]](https://cdn.slidesharecdn.com/ss_thumbnails/eng60115jan2013-1-130115121702-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Similar to Survey Methods - OIISDP 2015
Survey Methods - OIISDP 2015
- 1.
- 2.
- 3. 1. Measure behaviors, attitudes,or a construct 2. Collect relatively large amounts of data 3. Efficiency (cost vs. output)
- 4. Steps in surveyprocess • Step 1: Selecting/developing questions • Step 2: Select method of administration • Step 3: Pilot testing • Step 4: Sampling • Step 5: Writing up results
- 5.
- 7. What to lookfor in a good measure •Published •Used in a similar study •Good psychometric properties •Internal consistency estimates (𝛂 >.80) •Good factor structure •Evidence of validity
- 8. What is FactorAnalysis? •Factor analysis is all about simplification •Allows us to understand large quantities of observable variables in terms of a smaller number of unobservable variables •These unobservable variables are called “latent variables”
- 9. What is HomerSimpson like?
- 10. Homer likes tostick his hand in beehives Homer encourages his daughter to smoke Homer is distracted during a reactor meltdown • These are all directly observable phenomena • It might be easier just to say he’s stupid • Stupidity is a latent variable.
- 11. Factor Structure Factor Loadingsfor Exploratory Factor Analysis with Varimax Rotation of Facebook Activities Items Factor 1 Communicating Factor 2 Self-presentation Factor 3 Social Information Seeking Commenting 0.796 Liking 0.649 Sharing links 0.635 Status updates 0.628 Private messages 0.444 Chat 0.422 Tagging photos 0.826 Posting photos 0.811 Viewing photos 0.798 Checking up on 0.626 Creating or RSVP’ing to events 0.310
- 12.
- 13.
- 14.
- 15. Survey Mode Pros Cons Online Lowercost Quickly collect data Easy to modify Review results Branching Response rates Digital Inequalities Contacting participants Data loss Phone High response rates Interact with real person Diverse sampling High cost Take longer Cell phone numbers Mail Convenient Detailed responses Lower cost Efficient Take much longer Higher non completion Compete w/mail Project management
- 16. Levels of Measurement •Nominal:Names. Has no order. Assignment is arbitrary (1=East, 2=North, 3=South, etc.) •Ordinal: Has order, but interval between scale points may be uneven (1st place vs 2nd place runners compared to 50th and 51st place). Arithmetic operations are impossible with ordinal data. Can count and order. •Interval: Has order and equal intervals with an arbitrary zero point (years: year 1 AD is arbitrary - not when time began). Can add and subtract but not multiply and divide. •Ratio: Same as interval data with a true zero point (income: 0 income is truly no income). Can conduct all operations.
- 17. Measurement Level OperationsExamples Nominal No Ordering Sex (Male, Female) Ordinal Ordering, but not distance Student Class Standing (Freshman, etc.) Interval Distance, but not ratios GPA Ratio Ratios Number of Credit Hours
- 18.
- 19. Clarity • Provide clearinstructions • Emphasize rating scale • Never use compound questions • Will you allow “don’t know” option?
- 20. Questions Open Ended ClosedEnded Responses Greater variety of responses Interpretation Respondents interpret question the same way Missing Data More likely to skip Data Analyses No coding involved
- 21. Writing Questions • Avoidleading words • The government should force you to pay higher taxes. • Give mutually exclusive choices • What is your age? • 0-10 • 10-20 • 20-30
- 22. Writing Questions • Bedirect • How do you use the Internet? • Cover all possible answer choices • You indicated that you no longer use Facebook. Why not? • My parents joined Facebook • I like Instagram better • I lost my password
- 23. Likert Scales • Makesure they are ordinal and perhaps even interval • 5 or 7? Doesn’t matter • Balanced # of positives and negatives • Use scales with similar anchors • Reduces cognitive load but… • Spurious covariance
- 24. Question Order • Easy/fun/engagingquestions first • Sensitive & demographics last • Counterbalance
- 25.
- 26. Question Order • Easy/fun/engagingquestions first • Sensitive & demographics last • Counterbalance • Branch when you can
- 27.
- 28. Populations & Samples •Population:Entire collection of all of the data of interest •Sample: A subset of the population Population Sample
- 29. Choosing a Sample •Random: 1.Each person is chosen entirely by chance 2. Each member of the population has an equal chance of being included in the sample •Representative: The characteristics of the sample should match the characteristics of the population
- 30. Population: All collegestudents who use Facebook Sample: All students in an introductory psychology course who use Facebook
- 31. Sample Size •Larger samplesizes generally lead to increased precision when estimating parameters. •Sample must be large enough to detect differences in significance testing. •Calculate the sample size required to yield a certain power for a test, given a predetermined Type I error rate (ử). •Power: The probability that you will conclude there is no relationship when in fact there is.
- 32. Statistical Significance •Types ofError •Type I Error (significance): the chance you will conclude there is a relationship when there is not. •The chance that another random sample from the same population would result in a relationship as strong or stronger than the observed one, just by chance of sampling. Typically set at 5% (p < .05). •Type II Error (power): the chance you will conclude there is no relationship when in fact there is. Typically set at 20%, corresponding to a power level of .80. •Low power: where relationships which are real cannot be found to be significant (usually because sample size is too small). •High power: where even trivially small relationships are found significant (because sample size is excessive).
- 34. Choosing your sample •Calculatethe sample size required to yield a certain power for a test, given a predetermined Type I error rate (ử). •Figure out how to obtain a sample that is representative of your population of interest. •Randomly sample your population. •Simple random sampling: when you have a list which approximates all members of the population, then you draw from that list using a random number generator.
- 35.
- 36.
- 37. and so do effectsizes
- 38. Effect Size •A measureof how strongly the independent variables affect the dependent variables. • p-value is not effect size! •Cohen suggested: •Small: 0.01 •Medium: 0.059 •Large: 0.138 “These results were highly significant”
- 39.
- 40. •Understand your population •Offercongruent incentives •Appeal to intrinsic altruism •Personalize
- 41.
- 42.
- 44.
- 45.
- 46. 15% of respondents engagein sporadic careless responding
- 47.
- 48. What does actof giving survey do to future evaluations?
- 50.
- 52.