Your SlideShare is downloading. ×
0
Lecture 2 Survey Research & Design in Psychology James Neill,  2009 Introduction to  Survey Design
Overview <ul><li>The research process
Survey construction
Sampling
Levels of measurement
Measurement error </li></ul>
The research process 1. Establish  need for info/  research 2. Problem definition/ Hypotheses 3. Research design 4. Sampli...
The research process
Survey construction: Overview <ul><li>What is a survey?
Types of questionnaires
Questionnaire development
Writing questions
Types of questions
Response formats    LOM
Survey formatting </li></ul>
<ul><li>A standardised stimulus
A measuring instrument
A way of converting fuzzy psychological stuff  into hard data  for analysis </li></ul>What is a survey?
Types of surveys Types of surveys Self - administered Interview - administered Postal Delivered and collected Telephone Fa...
Questionnaire development 1. Formulate generic  questionnaire 2. Expand  the  questionnaire Turn into separate sections ba...
Writing questions - Dos <ul><li>Define  target constructs
Check related research & questionnaires
Draft items  (for important, fuzzy constructs aim to have multiple indicators)
Pre-test & revise </li></ul>
Writing questions - Dos <ul><li>Focus directly on topic/issue
Be clear
Be brief
Avoid big words
Use simple and correct grammar </li></ul>
Writing questions – Don'ts Inapplicable  –  must apply to all respondents Over-demanding  –  e.g., recall of detail or tim...
Writing questions – Don'ts Double-barrelled  -  e.g., “Do you think speed limits should be  lowered for cars & trucks?” Le...
Response biases <ul><li>Social desirability
Acquiescence </li><ul><li>yea- and nay-saying </li></ul><li>Self-serving bias
Order effects </li></ul>
Demand characteristics Interview <ul><li>High demand characteristics
Can elicit richer information </li></ul>Questionnaire <ul><li>Lower demand characteristics
Information may be less rich </li></ul>
Objective questions <ul><li>A verifiably true answer (i.e., factual information) exists for each unit.
The question could be accurately answered by an observer. </li></ul>e.g., How times during 2008 did you visit a G.P.?  ___...
Subjective questions <ul><li>Asks about fuzzy personal perceptions.
There is no “true”, factual answer.
Many possible answers per unit.
Can't be accurately answered by an observer. e.g., </li></ul>Think about the visits you made to a G.P. during 2008.  How w...
Objective vs. subjective questions <ul><li>Both types of questions may be appropriate; depends on the purpose of the study.
One criticism of this distinction: There is no such thing as “objective” and that  all responses  are subjective. </li></ul>
Accuracy of recall  decreases over time
Open-ended questions <ul><li>Rich information can be gathered
Useful for descriptive, exploratory work
Difficult and subjective to analyse
Time consuming </li></ul>
Open-ended questions:  Examples What are the main issues you are currently facing in your life? How many hours did you spe...
Closed-ended questions <ul><li>Important information may be lost forever
Useful for hypothesis testing
Easy and objective to analyse
Time efficient </li></ul>
How many hours did you spend studying this week?    less than 5 hours    > 5 to 10 hours    > 10 to 20 hours    more t...
Closed-ended questions:  Example (multiple response) What are the main issues you are currently facing in your life?  ( ti...
Closed-ended rating scales <ul><li>Dichotomous
Multichotomous
Verbal frquency scale
The list (multiple response)
Ranking
Likert scale
Graphical rating scale
Semantic differential
Non-verbal (idiographic) </li></ul>
Dichotomous 2 response options e.g.,  Excluding this trip, have you visited Canberra in the previous five years? __  Yes  ...
Multichotomous How many hours did you spend studying this week? __   less than 5 hours __   > 5 to 10 hours __   > 10 to 2...
Multichotomous More than two possible answers e.g., What type of attractions in your current trip to Canberra most appeal ...
Verbal frequency scale Over the past month, how often have you argued with your intimate partner? 1. All the time 2. Fairl...
The list (multiple response) Provides a list of answers for respondents to choose from e.g., Tick any words or phrases tha...
The list (multiple-response) What are the main issues that you are currently facing in your life? (tick all that apply) __...
Ranking Helps to measure the relative importance of several items Rank the importance of these reasons for taking a holida...
Likert scale Measures strength of feeling or perception. Indicate your degree of agreement with this statement: “ I am an ...
Graphical rating scale How would you rate your enjoyment of the movie you just saw?  Mark with a cross (X) not enjoyable  ...
What is your view of smoking?  Tick to show your opinion. Bad  ___:___:___:___:___:___:___  Good Strong  ___:___:___:___:_...
Non-verbal scale Point to the face that shows how you feel about what happened to the toy. Also called an  idiographic sca...
Verbal frequency scale Over the past month, how often have you argued with your intimate partner? 1. All the time 2. Fairl...
Sensitivity & reliability <ul><li>Scale should be sensitive yet reliable.
Watch out for too few or too many options. </li></ul>
How many response options? <ul><li>Minimum = 2
Average = 3 to 9
Maximum = 10? </li></ul>Basic guide: 7 +/- 2 Number of response options?
AGREEMENT ABOUT SOMETHING 2-Categories DISAGREE    AGREE 3-Categories DISAGREE    NEUTRAL   AGREE 4-Categories STRONGLY  M...
Watch out for too many or too few response options “ Capital punishment should be reintroduced for serious crimes ” 1 = Ag...
Example: How could  this question be improved? How old are you? ___ 18-20 ___ 20-22 ___ 22-30 ___ 30 and over
Are you satisfied with your marriage and your job? __________________________ Example: How could  this question be improved?
You didn’t think the food was very good, did you? _____ Yes _____ No Example: How could  this question be improved?
Environmental issues have become increasingly important in choosing hotels. Are environmental considerations an important ...
What information sources did you use to locate your restaurant for today’s meal? (please tick appropriate spaces)  ____ ye...
Comparison of  data collection methods Alreck & Settle (1995; 32) Alreck and Settle (1995:32)
Sampling
Sampling: Overview <ul><li>Sampling terminology
What is sampling?
Why sample?
Sampling methods
Example: Shere Hite’s survey </li></ul>
Sampling terminology <ul><li>Target population </li><ul><li>To whom you wish to generalise </li></ul><li>Sampling frame </...
What is sampling? “Sampling is the process of selecting units (e.g., people, organizations) from a population of interest ...
Why sample? <ul><li>Reduces cost, time, sample size etc.
If the sample is representative, the sample data allows inferences to be drawn about the total population. </li></ul>
Representativeness of a sample depends on: <ul><li>Adequacy of sampling frame
Sampling method
Adequacy of sample size
Response rate – both the % & representativeness of people in sample who actually complete survey </li></ul>It is better to...
Sampling methods Probability  sampling  <ul><li>Random
Systematic
Cluster </li><ul><li>Multi-Stage Cluster </li></ul></ul>Non-probability  sampling <ul><ul><li>Quota
Convenience
Snowball </li></ul></ul>
Random/probability sampling <ul><li>Each unit has an equal chance of selection
Selection occurs entirely by random chance
Upcoming SlideShare
Loading in...5
×

Survey Design Ii 1204634497987472 5

2,017

Published on

1 Comment
3 Likes
Statistics
Notes
No Downloads
Views
Total Views
2,017
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
146
Comments
1
Likes
3
Embeds 0
No embeds

No notes for slide
  • 7126/6667 Survey Research &amp; Design in Psychology Semester 1, 2009, University of Canberra, ACT, Australia James T. Neill http://ucspace.canberra.edu.au/display/7126 http://www.slideshare.net/jtneill/survey-design-ii Image sources: Note from Wikicommons GFDL Photo from http://www.flickr.com/photos/31910792@N05/3163866325/ CC-by-A 2.0 by jamesdal10 http://www.flickr.com/photos/31910792@N05/
  • Transcript of "Survey Design Ii 1204634497987472 5"

    1. 1. Lecture 2 Survey Research & Design in Psychology James Neill, 2009 Introduction to Survey Design
    2. 2. Overview <ul><li>The research process
    3. 3. Survey construction
    4. 4. Sampling
    5. 5. Levels of measurement
    6. 6. Measurement error </li></ul>
    7. 7. The research process 1. Establish need for info/ research 2. Problem definition/ Hypotheses 3. Research design 4. Sampling/ Data collection 5. Data analysis 6. Reporting
    8. 8. The research process
    9. 9. Survey construction: Overview <ul><li>What is a survey?
    10. 10. Types of questionnaires
    11. 11. Questionnaire development
    12. 12. Writing questions
    13. 13. Types of questions
    14. 14. Response formats  LOM
    15. 15. Survey formatting </li></ul>
    16. 16. <ul><li>A standardised stimulus
    17. 17. A measuring instrument
    18. 18. A way of converting fuzzy psychological stuff into hard data for analysis </li></ul>What is a survey?
    19. 19. Types of surveys Types of surveys Self - administered Interview - administered Postal Delivered and collected Telephone Face to face structured interview Web-based
    20. 20. Questionnaire development 1. Formulate generic questionnaire 2. Expand the questionnaire Turn into separate sections based on study objectives. Draft qs & response formats 4. Finalise questionnaire & implement Question order & funnel qs 3. Pre-test, pilot test, & redraft
    21. 21. Writing questions - Dos <ul><li>Define target constructs
    22. 22. Check related research & questionnaires
    23. 23. Draft items (for important, fuzzy constructs aim to have multiple indicators)
    24. 24. Pre-test & revise </li></ul>
    25. 25. Writing questions - Dos <ul><li>Focus directly on topic/issue
    26. 26. Be clear
    27. 27. Be brief
    28. 28. Avoid big words
    29. 29. Use simple and correct grammar </li></ul>
    30. 30. Writing questions – Don'ts Inapplicable – must apply to all respondents Over-demanding – e.g., recall of detail or time-consuming, unecessary questioning Ambiguous – meaning must be clear to all respondents Double negative – e.g., Do you not approve of tax reforms?
    31. 31. Writing questions – Don'ts Double-barrelled - e.g., “Do you think speed limits should be lowered for cars & trucks?” Leading - e.g., “don’t you see some danger in the new policy?” Loaded – e.g., “Do you advocate a lower speed limit to save human lives?” vs “Does traffic safety require a lower speed limit?”
    32. 32. Response biases <ul><li>Social desirability
    33. 33. Acquiescence </li><ul><li>yea- and nay-saying </li></ul><li>Self-serving bias
    34. 34. Order effects </li></ul>
    35. 35. Demand characteristics Interview <ul><li>High demand characteristics
    36. 36. Can elicit richer information </li></ul>Questionnaire <ul><li>Lower demand characteristics
    37. 37. Information may be less rich </li></ul>
    38. 38. Objective questions <ul><li>A verifiably true answer (i.e., factual information) exists for each unit.
    39. 39. The question could be accurately answered by an observer. </li></ul>e.g., How times during 2008 did you visit a G.P.? ______
    40. 40. Subjective questions <ul><li>Asks about fuzzy personal perceptions.
    41. 41. There is no “true”, factual answer.
    42. 42. Many possible answers per unit.
    43. 43. Can't be accurately answered by an observer. e.g., </li></ul>Think about the visits you made to a G.P. during 2008. How well did you understand the medical advice you received? perfectly very well reasoably poorly not at all
    44. 44. Objective vs. subjective questions <ul><li>Both types of questions may be appropriate; depends on the purpose of the study.
    45. 45. One criticism of this distinction: There is no such thing as “objective” and that all responses are subjective. </li></ul>
    46. 46. Accuracy of recall decreases over time
    47. 47. Open-ended questions <ul><li>Rich information can be gathered
    48. 48. Useful for descriptive, exploratory work
    49. 49. Difficult and subjective to analyse
    50. 50. Time consuming </li></ul>
    51. 51. Open-ended questions: Examples What are the main issues you are currently facing in your life? How many hours did you spend studying this week? _________
    52. 52. Closed-ended questions <ul><li>Important information may be lost forever
    53. 53. Useful for hypothesis testing
    54. 54. Easy and objective to analyse
    55. 55. Time efficient </li></ul>
    56. 56. How many hours did you spend studying this week?  less than 5 hours  > 5 to 10 hours  > 10 to 20 hours  more than 20 hours Closed-ended questions: Example (multichotomous)
    57. 57. Closed-ended questions: Example (multiple response) What are the main issues you are currently facing in your life? ( tick all that apply)  financial  physical / health  academic  employment / unemployment  intimate relations  social relations  other (please specify) ________________________________
    58. 58. Closed-ended rating scales <ul><li>Dichotomous
    59. 59. Multichotomous
    60. 60. Verbal frquency scale
    61. 61. The list (multiple response)
    62. 62. Ranking
    63. 63. Likert scale
    64. 64. Graphical rating scale
    65. 65. Semantic differential
    66. 66. Non-verbal (idiographic) </li></ul>
    67. 67. Dichotomous 2 response options e.g., Excluding this trip, have you visited Canberra in the previous five years? __ Yes __ No Provides the simplest type of quantification
    68. 68. Multichotomous How many hours did you spend studying this week? __ less than 5 hours __ > 5 to 10 hours __ > 10 to 20 hours __ more than 20 hours
    69. 69. Multichotomous More than two possible answers e.g., What type of attractions in your current trip to Canberra most appeal to you? __ historic buildings __ museum/art galleries __ parks and gardens
    70. 70. Verbal frequency scale Over the past month, how often have you argued with your intimate partner? 1. All the time 2. Fairly often 3. Occasionally 4. Never 5. Doesn’t apply to me at the moment
    71. 71. The list (multiple response) Provides a list of answers for respondents to choose from e.g., Tick any words or phrases that describe your perception of Canberra as a travel destination: __ Exciting __ Important __ Boring __ Enjoyable __ Interesting __ Historical
    72. 72. The list (multiple-response) What are the main issues that you are currently facing in your life? (tick all that apply) __ financial __ physical / health __ academic __ employment / unemployment __ relationships __ other (please specify)
    73. 73. Ranking Helps to measure the relative importance of several items Rank the importance of these reasons for taking a holiday to Canberra (from 1 (most) to 4 (least)): __ to visit friends and relatives __ for business __ for educational purposes __ for holiday/ sightseeing
    74. 74. Likert scale Measures strength of feeling or perception. Indicate your degree of agreement with this statement: “ I am an adventurous person.” (circle the best response for you) 1 2 3 4 5 strongly disagree neutral agree strongly disagree agree 1 2 3 4 5 strongly agree neutral disagree strongly agree disagree
    75. 75. Graphical rating scale How would you rate your enjoyment of the movie you just saw? Mark with a cross (X) not enjoyable very enjoyable
    76. 76. What is your view of smoking? Tick to show your opinion. Bad ___:___:___:___:___:___:___ Good Strong ___:___:___:___:___:___:___ Weak Masculine ___:___:___:___:___:___:___ Feminine Unattractive ___:___:___:___:___:___:___ Attractive Passive ___:___:___:___:___:___:___ Active Semantic differential
    77. 77. Non-verbal scale Point to the face that shows how you feel about what happened to the toy. Also called an idiographic scale .
    78. 78. Verbal frequency scale Over the past month, how often have you argued with your intimate partner? 1. All the time 2. Fairly often 3. Occasionally 4. Never 5. Doesn’t apply to me at the moment
    79. 79. Sensitivity & reliability <ul><li>Scale should be sensitive yet reliable.
    80. 80. Watch out for too few or too many options. </li></ul>
    81. 81. How many response options? <ul><li>Minimum = 2
    82. 82. Average = 3 to 9
    83. 83. Maximum = 10? </li></ul>Basic guide: 7 +/- 2 Number of response options?
    84. 84. AGREEMENT ABOUT SOMETHING 2-Categories DISAGREE AGREE 3-Categories DISAGREE NEUTRAL AGREE 4-Categories STRONGLY MILDLY MILDLY STRONGLY DISAGREE DISAGREE AGREE AGREE 5-Categories STRONGLY MILDLY MILDLY STRONGLY DISAGREE DISAGREE NEUTRAL AGREE AGREE Number of response options? Likert scale example
    85. 85. Watch out for too many or too few response options “ Capital punishment should be reintroduced for serious crimes ” 1 = Agree 2 = Disagree 1 = Very, Very Strongly Agree 7 = Slightly Disagree 2 = Very Strongly Agree 8 = Disagree 3 = Strongly Agree 9 = Strongly Disagree 4 = Agree 10 = V. Strongly Disagree 5 = Slightly Agree 11 = V, V Strongly Disagree 6 = Neutral
    86. 86. Example: How could this question be improved? How old are you? ___ 18-20 ___ 20-22 ___ 22-30 ___ 30 and over
    87. 87. Are you satisfied with your marriage and your job? __________________________ Example: How could this question be improved?
    88. 88. You didn’t think the food was very good, did you? _____ Yes _____ No Example: How could this question be improved?
    89. 89. Environmental issues have become increasingly important in choosing hotels. Are environmental considerations an important factor when deciding on your choice of hotel accommodation? ____ Yes ____ No Example: How could this question be improved?
    90. 90. What information sources did you use to locate your restaurant for today’s meal? (please tick appropriate spaces) ____ yellow pages ____ Internet ____ word of mouth Example: How could this question be improved?
    91. 91. Comparison of data collection methods Alreck & Settle (1995; 32) Alreck and Settle (1995:32)
    92. 92. Sampling
    93. 93. Sampling: Overview <ul><li>Sampling terminology
    94. 94. What is sampling?
    95. 95. Why sample?
    96. 96. Sampling methods
    97. 97. Example: Shere Hite’s survey </li></ul>
    98. 98. Sampling terminology <ul><li>Target population </li><ul><li>To whom you wish to generalise </li></ul><li>Sampling frame </li><ul><li>Those who have a chance to be selected </li></ul><li>Sample </li><ul><li>Those who were selected and responsed </li></ul><li>Representativeness </li><ul><li>The extent to which the sample is a good indicator of the target population </li></ul></ul>
    99. 99. What is sampling? “Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.” - Trochim (2006)
    100. 100. Why sample? <ul><li>Reduces cost, time, sample size etc.
    101. 101. If the sample is representative, the sample data allows inferences to be drawn about the total population. </li></ul>
    102. 102. Representativeness of a sample depends on: <ul><li>Adequacy of sampling frame
    103. 103. Sampling method
    104. 104. Adequacy of sample size
    105. 105. Response rate – both the % & representativeness of people in sample who actually complete survey </li></ul>It is better to have a small, representative sample than a large, unrepresentative sample.
    106. 106. Sampling methods Probability sampling <ul><li>Random
    107. 107. Systematic
    108. 108. Cluster </li><ul><li>Multi-Stage Cluster </li></ul></ul>Non-probability sampling <ul><ul><li>Quota
    109. 109. Convenience
    110. 110. Snowball </li></ul></ul>
    111. 111. Random/probability sampling <ul><li>Each unit has an equal chance of selection
    112. 112. Selection occurs entirely by random chance
    113. 113. Also called representative sampling </li></ul>
    114. 114. <ul>Simple random sampling </ul><ul><li>Everyone in the target population has an equal chance of selection
    115. 115. Useful if clear study area or population is identified
    116. 116. Similar to a lottery: </li><ul><li>List of names are assigned #s and randomly select #s of respondents
    117. 117. Randomly select # through table of random #s or by computer </li></ul></ul>
    118. 118. Systematic random sampling <ul><li>Selecting without first numbering
    119. 119. Respondents (units) selected from a list/file.
    120. 120. Useful when survey population is similar e.g. List of students
    121. 121. Select sample at regular intervals from the population e.g., every 5 th person on a list, starting at a random number between 1 and 5 </li></ul>
    122. 122. <ul>Stratified random sampling </ul><ul><li>Sub-divide population into strata (e.g., by gender, age, or location)
    123. 123. Then random selection from within each stratum
    124. 124. Improves representativeness
    125. 125. e.g., Telephone interviews using post-code strata </li></ul>
    126. 126. <ul>Non-random / non-probability </ul><ul><li>Also called purposive or judgemental sampling
    127. 127. Useful for exploratory research and case study research
    128. 128. Able to get large sample size quickly
    129. 129. Limitations include potential bias and non-representativeness </li></ul>
    130. 130. <ul>Convenience sampling </ul><ul><li>Sampling is by convenience rather than randomly
    131. 131. Due to time/financial constraints
    132. 132. e.g. surveying all those at a tourist attraction over one weekend </li></ul>
    133. 133. <ul>Purposive sampling </ul>Respondents selected for a particular purpose e.g., because they may be “typical” respondents <ul><li>e.g., select sample of tourists aged 40-60 as this is the typical age group of visitors to Canberra
    134. 134. e.g., Frequent flyers to contact regarding service quality in an airline setting </li></ul>
    135. 135. <ul>Snowball sampling </ul><ul><li>Useful for difficult to access populations e.g., illegal immigratnts, drug users
    136. 136. Respondents recommend other respondents
    137. 137. e.g., in studying ecstasy users, gain trust of a few potential respondents and ask them to recommend the researcher to other potential respondents </li></ul>
    138. 138. Sampling process <ul><li>Identify target population and sampling frame
    139. 139. Select sampling method
    140. 140. Calculate sample size for desired power.
    141. 141. Maximise return rate </li></ul>
    142. 142. Summary of sampling strategy <ul><li>Identify target population and sampling frame
    143. 143. Selection sampling method
    144. 144. Calculate required sample size
    145. 145. Maximise return rate </li></ul>
    146. 146. Sampling Example: Shere Hite ‘American Sexology’
    147. 147. Hite's survey of American male-female relations (early 1980's) <ul><li>Shere Hite ‘doyenne of sex polls’
    148. 148. Media furors & worldwide attention
    149. 149. 127-item questionnaire about marriage & relations between sexes
    150. 150. Sample: 4500 USA women, 14 to 85 years
    151. 151. Conclusion: Society and men need to change to improve lives of women </li></ul>
    152. 152. Some of Hite’s findings about American women.... <ul><li>Only 13% married for 2+years were still in love
    153. 153. 70% married for 5+ years were having affairs... </li><ul><li>usually more for 'emotional closeness’ than sex
    154. 154. 76% of these women did not feel guilty </li></ul><li>87% had a closer female friend than husband
    155. 155. 98% wanted “basic changes” to love relationships
    156. 156. 84% were emotionally unsatisfied
    157. 157. 95% reported emotional & psychological harassment from their men </li></ul>
    158. 158. Some of the critical comments.... She goes in with prejudice & comes out with a statistic. The survey often seems merely to provide an occasion for the author’s own male-bashing diatribes. Hite uses statistics to bolster her opinion that American women are justifiably fed up with American men.
    159. 159. Hite's response rate & selection bias <ul><li>100,000 questionnaires were sent to a variety of women’s groups (feminist organisations, church groups, garden clubs etc.)
    160. 160. 4,500 replied (4.5% return rate) </li></ul>
    161. 161. “ We get pretty nervous if respondents in our survey go under 70%. Respondents to surveys differ from nonrespondents in one important way: they go to the trouble of filling out what in this case was a very long, complicated, and personal questionnaire.” - Regina Herzog, University of Michigan Institute for Social Research Hite's response rate & selection bias
    162. 162. <ul><li>Sample size – it's now big, it's how representative
    163. 163. Objectivity – watch out for manipulating the survey questions and results interpretation to suit your personal conjectures </li></ul>Lessons from Hite's male-female relations survey
    164. 164. Survey Format Checklist
    165. 165. Introduction or cover letter: <ul><li>Few will read it without good prompting and being easy-to-read </li></ul>Instructions <ul><li>Provides consistency - helps to ensure standard conditions across different administrations
    166. 166. Explain how to do the survey in a user-friendly manner </li></ul>Survey format checklist
    167. 167. Cover letter / ethics statement Outline details of research project e.g.,: <ul><li>Who are you? Are you bona fide?
    168. 168. Purpose of survey?
    169. 169. What's involved?
    170. 170. Explain any risks/costs/rewards
    171. 171. How will results be used?
    172. 172. Human ethics approval #
    173. 173. How is consent given / not given?
    174. 174. Voluntary - can choose not to continue anytime
    175. 175. More info: Complaints, how to obtain results, contact details etc. </li></ul>
    176. 176. Instructions: Example
    177. 177. <ul><li>Group like questions together
    178. 178. Order effects: </li><ul><li>Habituation
    179. 179. Fatigue
    180. 180. Minimise switching between response formats </li></ul></ul>Survey format checklist
    181. 181. Layout <ul><li>Font (type, size)
    182. 182. No. of pages
    183. 183. Margins
    184. 184. Double vs. single-siding
    185. 185. Colour, etc. </li></ul>Survey format checklist
    186. 186. Demographics <ul><li>single section, usually at beginning or end of questionnaire
    187. 187. only include relevant questions </li></ul>Survey format checklist
    188. 188. <ul><li>Space for comments?
    189. 189. Indicate the end
    190. 190. Say thanks!
    191. 191. Pre-test & pilot-test </li></ul>Survey format checklist
    192. 192. Pre- & pilot-testing <ul><li>Pre-test items on convenient others - ask for feedback
    193. 193. Revise items e.g., </li></ul><ul><ul><li>Which don’t apply to everybody
    194. 194. Are redundant
    195. 195. Are misunderstood
    196. 196. Are non-completed </li></ul></ul><ul><li>Reconsider ordering & layout
    197. 197. Pilot test on a small sample from the target population, analyse, & revise </li></ul>
    198. 198. Ethical issues: How to treat respondents <ul><li>Minimise risk / harm to respondents
    199. 199. Informed consent
    200. 200. Confidentiality / anonymity
    201. 201. No coercion
    202. 202. Minimal deceit
    203. 203. Fully debrief </li></ul>
    204. 204. <ul><li>Honour promises to provide respondents with research reports
    205. 205. Be aware of potential sources of bias/ conflicts of interest
    206. 206. Represent research literature fairly </li></ul>Ethical issues: Other
    207. 207. <ul><li>Don’t search data for pleasing findings
    208. 208. Acknowledge all sources
    209. 209. Don’t fake (or unfairly manipulate) data
    210. 210. Honestly report research findings </li></ul>Ethical issues: Other
    211. 211. Levels of Measurement = Type of Data Stevens (1946)
    212. 212. Levels of measurement <ul><li>N ominal / Categorical
    213. 213. O rdinal
    214. 214. I nterval
    215. 215. R atio </li></ul>
    216. 216. Discrete vs. continuous Discrete - - - - - - - - - - Continuous ___________
    217. 217. Each level has the properties of the preceding levels, plus something more!
    218. 218. Categorical / nominal <ul><li>Conveys a category label
    219. 219. (Arbitrary) assignment of #s to categories </li></ul>e.g. Gender <ul><li>No useful information, except as labels </li></ul>
    220. 220. Categorical / nomimal example: Phrenological labels
    221. 221. Ordinal / ranked scale <ul><li>Conveys order , but not distance </li></ul>e.g. in a race, 1st, 2nd, 3rd, etc. or ranking of favourites or preferences
    222. 222. Ordinal / ranked example: Ranked importance Rank the following aspects of the university according to what is most important to you (1 = most important through to 5 = least important)  Quality of the teaching and education  Quality of the social life  Quality of the campus  Quality of the administration  Quality of the university's reputation
    223. 223. Interval scale <ul><li>Conveys order & distance
    224. 224. 0 is arbitrary </li></ul>e.g., temperature (degrees C) <ul><li>Usually treat as continuous for > 5 intervals </li></ul>
    225. 225. Interval example: 8 point Likert scale
    226. 226. Ratio scale <ul><li>Conveys order & distance
    227. 227. Continuous, with a meaningful 0 point </li></ul>e.g. height, age, weight, time, number of times an event has occurred <ul><li>Ratio statements can be made </li></ul>e.g. X is twice as old (or high or heavy) as Y
    228. 228. Ratio scale: Time
    229. 229. Why do levels of measurement matter? Different analytical procedures are used for different levels of data. More powerful statistics can be applied to higher levels
    230. 230. Levels of measurement: Practice exam question Fill in all cells Descriptive Statistics Ratio Interval Ordinal / Rank Nominal / Categorical Graphs Examples Prop-erties Level
    231. 231. Golden rule of data analysis Choose analysis techniques to match : <ul><ul><li>Research questions / hypotheses
    232. 232. Type (level) of data </li></ul></ul>
    233. 233. Levels of measurement and data analysis categorical & ordinal DVs  non-parametric interval & ratio DVs  parametric
    234. 234. Levels of measurement and data analysis For more info, see an Inferential Statistics Decision-Making Tree
    235. 235. Parametric statistics = procedures which estimate PARAMETERS of a population, usually based on the normal distribution <ul><li>Any procedure which uses M , SD - e.g. t-tests, ANOVAs
    236. 236. Any procedure which uses r - e.g. bivariate correlation, linear regression </li></ul>
    237. 237. Non-parametric statistics (Distribution-free Tests) = procedures which do not rely on estimates of population parameters <ul><li>Any procedure which uses frequency - e.g. sign test, chi-squared
    238. 238. Any procedure which uses rank order - e.g. Mann-Whitney U test, Wilcoxon matched-pairs signed-ranks test </li></ul>
    239. 239. <ul><li>More powerful
    240. 240. More sensitive to violations of assumptions </li></ul>Parametric statistics
    241. 241. Measurement Error
    242. 242. Measurement error <ul><li>Observed score = </li></ul>true score + measurement error <ul><li>Measurement error = </li></ul>systematic error + random error <ul><li>Any deviation from the true value caused by the measurement procedure. </li></ul>
    243. 243. Sources of measurement error Non-sampling (e.g., unreliable or invalid tests) Sampling (e.g., non-rep. sample) Personal bias (e.g., researcher favours a hypothesis) Paradigm (e.g., Western focus on individualism)
    244. 244. To minimise measurement error Use well designed measures: <ul><li>Multiple indicators
    245. 245. Sensitive to target constructs
    246. 246. Clear wording on questions/instructions </li></ul>
    247. 247. Reduce demand effects: <ul><li>Train interviewers
    248. 248. Use standard protocol </li></ul>To minimise measurement error
    249. 249. To minimise measurement error Maximise response rate: <ul><li>Pre-survey contact
    250. 250. Minimise length / time / hassle
    251. 251. Offer rewards / incentives
    252. 252. Coloured paper
    253. 253. Call backs / reminders </li></ul>
    254. 254. Ensure administrative accuracy <ul><li>Set up efficient coding, with well-labelled variables
    255. 255. Check data </li></ul>To minimise measurement error
    256. 256. Summary - 1 <ul><li>Survey research has developed into a popular research method since the 1940's.
    257. 257. A survey is a standardised stimulus designed to convert fuzzy psychological phenomenon into hard data. </li></ul>
    258. 258. Summary - 2 <ul><li>Survey development - types of questions and response formats.
    259. 259. Sampling - probability & non-prob.
    260. 260. Levels of measurement & parametric / non-parametric stats
    261. 261. Ethical considerations
    262. 262. Sources of measurement error </li></ul>
    263. 263. References Alreck, P. & Settle, R. (1995). The survey research handbook (2 nd ed.). New York: Irwin. Stevens, S.S. (1946). On the theory of scales of measurement. Science , 103 , 677-680. Trochim, W. M. K. (2006). Sampling . In Research Methods Knowledge Base . Wikipedia (2009). Shere Hite - Methodology .
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×