Survey Instrument<br />IA4340-1[Tue 1,2,3]D406<br />IA4340-2[Tue 5,6,7]D325<br />Visual Design/2011 Fall Semester<br />Wee...
설문 문항 만들기 [DON] Chapter 2. Writing Questions[CON] Chapter 2. The Experimental Evidence[SAL] Chapter 6. Writing Good Questi...
Goal of Writing Questions<br />Developing a query that<br />Every potential respondent will interpret in the same way<br /...
Question 1<br />Dose the question require an answer?<br />In order for an inquiry to constitute a survey question, it must...
Question 2<br />To what extent do survey recipients already have an accurate, ready-made answer for the question they are ...
Question 3<br />Can people accurately recall & report past behaviors?<br />People who write surveys want respondents to pr...
Question 4 ,5 & 6<br />Is the respondent willing to reveal the requested information?<br />The fact that respondents know ...
Question 7<br />Is survey information being collected by more than one mode?<br />Data collected by each mode needs to be ...
Question 8<br />Changing a question acceptable to the survey sponsor?<br />Questions with recognized defects cannot always...
Writing Good Questions<br />What kind of information are you looking for?<br />Two types of information<br />Behavior or A...
Question Structure<br />Open-Ended Questions<br />Open-ended questions do not provide choices from which to select an answ...
Question Structure<br />Open-Ended Questions<br />The answer depends on the extent to which respondents are willing to thi...
Question Structure<br />Open-Ended Questions<br />This type of question is useful,<br />When researchers have little prior...
Question Structure<br />Closed-Ended Question with Ordered Categories<br />This type of question is useful,<br />When one ...
Question Structure<br />Closed-Ended Question with Ordered Categories<br />This type of question is useful,<br />When a su...
Question Structure<br />Closed-Ended Question with Unordered Categories<br />People are asked to choose from among discret...
Question Structure<br />Partially Closed-Ended Question<br />Compromising between the open- and the closed-ended structure...
Writing Good Questions<br />Why measuring attitudes and beliefs requires special attention?<br />Attitudes and beliefs are...
Writing Good Questions<br />Does the order of response choices matter?<br />Example<br />In mail survey, people are somewh...
Principles for Writing Good Questions<br />Specific questions are better than general ones<br />The same question sometime...
Principles for Writing Good Questions<br />Chose simple over specialized words<br />Shorter & more easily understood word<...
Principles for Writing Good Questions<br />Chose as few words as possible to pose<br />Redundancy across many questions is...
Principles for Writing Good Questions<br />Use complete sentences to ask questions<br />Avoid vague quantifiers when more ...
Principles for Writing Good Questions<br />Offer a “no opinion” or “don’t know” option<br />Distinguish undecided from neu...
Principles for Writing Good Questions<br />Eliminate check-all-that-apply question formats<br />Primary effect<br />Tenden...
Principles for Writing Good Questions<br />Provide appropriate time referents<br />Be sure each question is technically ac...
Summary<br />Principles for Writing Good Questions<br />Encouraging thoughtful answers<br />Cognitive design<br />Overcomi...
설문지 만들기[DON] Chapter 3. Constructing the questionnaire[COV] Chapter 3. The tools at hand<br />
Criteria for Design	<br />Importance of questionnaire design<br />To reduce nonresponse<br />Respondent-friendly questionn...
Alternative questionnaire formats and why a booklet is preferred<br />Unacceptable formats<br />Printing on both sides of ...
Criteria For Ordering The Questions<br />Possible order effects between opinion questions<br />Norm of evenhandedness, a v...
Criteria For Ordering The Questions<br />Choose the first question carefully<br />Should be clearly apply to everyone<br /...
Principle 3.6<br />Begin asking questions in the upper quadrant<br />The ideal place to start asking a question is near th...
Principle 3.9<br />Number questions consecutively and simply, from beginning to end<br />Example<br />P1, P2, H1, H2only ...
Principle 3.16<br />Do not place instructions in a separate instruction book or in a separate section of the questionnaire...
Principle 3.18	<br />When shaded background fields are used, identification of all answer spaces in white helps to reduce ...
Principle 3.22<br />Vertical alignment of question subcomponents among consecutive questions eases the response task<br />...
Maintain spacing between answer choices that is consistent with measurement intent</li></ul>Principles for constructing qu...
Principle 3.25<br />Maintain consistency throughout a questionnaire in the direction scales are displayed<br /><ul><li>Pri...
Use shorter lines to prevent word form being skipped.attentivelyon about 8 to 10 characters of text at
The preference for using two column formats on 8 ½ X11“ pages instead of entire page</li></ul>Principles for constructing ...
Step 3<br />Developing additional visual navigational guides, the aim of which is to <br />interrupt established navigatio...
Principle 3.28<br />Words and phrases that introduce important, but easy to miss, changes in respondent expectations shoul...
Designing the questionnaire cover pages<br />Questionnaire cover designs<br />Improve response rate<br />The front cover<b...
Designing the questionnaire cover pages<br />The back cover<br />Should be kept simple<br />Consisted of an invitation to ...
Pretesting : Four sequential stages<br />Stage 1<br />Review by knowledgeable colleagues and analysts<br />To elicit sugge...
Pretesting : Four sequential stages<br />Stage 3<br />A small pilot study<br />Can estimate to be made for response rates,...
The purpose of pretests<br />Testing questions<br />Variation<br />Testing items foran acceptable level of variationin the...
The purpose of pretests<br />Pretesting the questionnaire<br />“flow” and naturalness of the sections<br />A matter of int...
Upcoming SlideShare
Loading in...5
×

7강 survey

972

Published on

Published in: Education, Spiritual, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
972
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

7강 survey

  1. 1. Survey Instrument<br />IA4340-1[Tue 1,2,3]D406<br />IA4340-2[Tue 5,6,7]D325<br />Visual Design/2011 Fall Semester<br />Week 07/UI Design<br />
  2. 2. 설문 문항 만들기 [DON] Chapter 2. Writing Questions[CON] Chapter 2. The Experimental Evidence[SAL] Chapter 6. Writing Good Questions<br />
  3. 3. Goal of Writing Questions<br />Developing a query that<br />Every potential respondent will interpret in the same way<br />Every potential respondent will be able to respondent<br />Every potential respondent will be willing to answer<br />Once a working draft of proposed survey questions and response choices has been prepared, ask each of the following 8 inquiries about each of the proposed survey questions<br />Answers to these questions will help diagnose problems and guide you towards the structural and wording decisions that are appropriate for your study<br />
  4. 4. Question 1<br />Dose the question require an answer?<br />In order for an inquiry to constitute a survey question, it must require an answer from each person to whom the question is asked<br />Use of the introductory words “if” or “when”, invites some respondents not to provide an answer<br />Especially, for us to be able to estimate the distribution of a characteristic in the sample and the population, respondents must be given the opportunity to answer every question they are asked<br />Example 1) Ifyou fixed dinner at home last night, did you eat meat as part of that meal?<br />4<br />
  5. 5. Question 2<br />To what extent do survey recipients already have an accurate, ready-made answer for the question they are being asked to report?<br />If people do not have a ready-made answer to a question, getting an accurate answer becomes more difficult<br />The vaguer the question, the vaguer the categories, and the more remote these items are from people’s experiences, the more likely a question is to produce inconsistent responses if we ask the same person to answer this question at different times<br />When respondents have ready-made answer,<br />Example) How old are you? <br />When respondents don’t have ready-made answer,<br />Example) Tall people are more likely to be elected President of the United States, <br />5<br />
  6. 6. Question 3<br />Can people accurately recall & report past behaviors?<br />People who write surveys want respondents to provide far more detail about past behaviors than can be recalled<br />Determining whether people are able to recall information needed for answering each proposed survey question is important<br />Keeping recall simple and related to recent events helps to produce high quality survey data<br />6<br />
  7. 7. Question 4 ,5 & 6<br />Is the respondent willing to reveal the requested information?<br />The fact that respondents know an answer dose not mean they are willing to provide it<br />Will the respondent feel motivated to answer each question?<br /> Motivation can be encouraged in many ways, ranging from incentives and follow-up reminders to respondent-friendly questionnaire design<br />Is the respondent’s understanding of response categories likely to be influenced by more than words?<br />The vaguer the question and answer categories, the greater the potential for measurement error<br />Category ranges, visual lay out…<br />7<br />
  8. 8. Question 7<br />Is survey information being collected by more than one mode?<br />Data collected by each mode needs to be comparable with that collected by another<br />Several remedial actions can be done to minimize mode effects (Ch. 6)<br />Restructuring questions & Changing question wording<br />Response Rate and Measurement Differences in Mixed Mode Surveys respondent (Dillman, Sangster, Tarnai, and Rockwood, 1996).<br />8<br />
  9. 9. Question 8<br />Changing a question acceptable to the survey sponsor?<br />Questions with recognized defects cannot always be changed<br />Example) Time-series Data<br />It is important to ask sponsors whether question that appear troublesome are subject to change and if so how much.<br />Summary<br />How specific the questions should be<br />Whether the questions will produce credible information<br />Whether respondents are able to answer the questions<br />Whether respondents will be willing to provide the information<br />9<br />
  10. 10. Writing Good Questions<br />What kind of information are you looking for?<br />Two types of information<br />Behavior or Attribute: What people do or what they are<br />Attitude or Belief: What people say that they want or what they think is true<br />Deciding which question structure is most useful<br />Each type of question structure performs a role that no other type can achieve as well, making an understanding of question structure a fundamental tool for drafting acceptable survey questions <br />Open-Ended Questions<br />Closed-Ended Question with Ordered Categories<br />Closed-Ended Question with Unordered Categories<br />Partially Closed-Ended Question<br />10<br />
  11. 11. Question Structure<br />Open-Ended Questions<br />Open-ended questions do not provide choices from which to select an answer<br />Instead, respondents must formulate an answer in their own words<br />Disadvantage<br />They are difficult for respondents<br />They produce many different responses and only a few mentions of any one topic<br />They rarely provide accurate measurements or consistent, comparable information across the whole sample (Quiz 4)<br />They are extremely difficult to code<br />
  12. 12. Question Structure<br />Open-Ended Questions<br />The answer depends on the extent to which respondents are willing to think hard about the question and write a complete answer<br />People don’t necessarily have a ready-made answer and the answers are prone to considerable unintentional error<br />Providing blank spaces followed by the unit for reporting<br />12<br />
  13. 13. Question Structure<br />Open-Ended Questions<br />This type of question is useful,<br />When researchers have little prior knowledge about a topic and therefore cannot specify response choices<br />When the main goal is to give survey respondents a chance to state strong opinions, or let researchers know what has been overlooked<br />When they immediately follow a closed-ended question and ask respondents to explain why they selected a particular answer<br />When people are being asked to estimate a routine behavior and are unlikely to know an exact number<br />In which a precise piece of information is needed and can be easily recalled without a list of answer choices<br />When researchers are interest in the sensitive or disapproved behavior<br />
  14. 14. Question Structure<br />Closed-Ended Question with Ordered Categories<br />This type of question is useful,<br />When one has a well-defined concept for which an evaluative response is wanted, unencumbered by thoughts of alternative or competing ideas<br />Example)<br />Using the mobile internet service increases my productivity.<br />1 2 3 4 5 6 7<br />Learning how to use the mobile internet service was easy for me.<br />1 2 3 4 5 6 7 <br />
  15. 15. Question Structure<br />Closed-Ended Question with Ordered Categories<br />This type of question is useful,<br />When a surveyor wants to obtain separate respondent evaluations of many different concept<br />Example) <br />The mobile Internet services fall into the following twocategories. <br />How important are the services for you, at present? <br />[Commerce]<br />1 2 3 4 5 6 7 <br />[Communication]<br />1 2 3 4 5 6 7 <br />
  16. 16. Question Structure<br />Closed-Ended Question with Unordered Categories<br />People are asked to choose from among discrete, unordered categories<br />Use the unordered format only when your knowledge of the subject allows you to list useful answer choices <br />This type of question is useful,<br />When a surveyor wants to obtain specific response:<br />16<br />
  17. 17. Question Structure<br />Partially Closed-Ended Question<br />Compromising between the open- and the closed-ended structures<br />Limiting choices to some alternatives means that some people will not be able to give the answer they want to provide<br />
  18. 18. Writing Good Questions<br />Why measuring attitudes and beliefs requires special attention?<br />Attitudes and beliefs are very difficult to measure because they are often imprecise, change form day to day, and may not be well thought out in advance of the survey<br />People do not “possess” attitudes and beliefs in the same way they “possess” attributes like age and gender<br />Advice<br />Avoid using extremely abstract questions<br />Use different kinds of question structure so that you rely on any single question<br />Scaling Technique: Using Multi-Item<br />18<br />
  19. 19. Writing Good Questions<br />Does the order of response choices matter?<br />Example<br />In mail survey, people are somewhat more likely to chose from among the first categories listed<br />In telephone and face-to-face interviews, they are more likely to choose from among the last<br />Researcher should be aware of how it might increase your measurement error<br />Try to keep your lists of answer choices from getting from long<br />Systematically vary the order<br />19<br />
  20. 20. Principles for Writing Good Questions<br />Specific questions are better than general ones<br />The same question sometimes means different things to different people<br />The more general the question, the wider the range of interpretations it may be given<br />Responses to general attitude items are poorer predictors of behavior than responses to specific attitude questions<br />By contrast, wording that is specific and concrete is more apt to communicate uniform meaning<br />The more specific an attitude item, the stronger the connection between attitudes and behaviors<br />
  21. 21. Principles for Writing Good Questions<br />Chose simple over specialized words<br />Shorter & more easily understood word<br />Example<br />Using combinations of shorter words<br />Example<br /> tired ← exhausted honest ← candid<br /> more important ← top priority free time ← leisure<br /> work ← employment brave ← courageous<br /> correct ← rectify <br />people who live here ← occupants of this house<br />your answer ← your response to this questionnaire<br />what you do after school ← post-school extracurricular activities<br />area of the country ← sub-national region<br />
  22. 22. Principles for Writing Good Questions<br />Chose as few words as possible to pose<br />Redundancy across many questions is a particularly strong indicator to respondents that it is okay to skip words, and may result in the rest of the sentence also being unevenly read<br />Do you strongly favor, somewhat favor, somewhat oppose, strongly oppose, or have no opinion on whether advertisers should be required to have advertising aimed at children approved by a national board?<br /> -> Which of the following best describes the extent to which you favor or oppose requiring advertisers to have advertising aimed at children approved by a national review board?<br />Example) Please indicate the extent to which you agree with each statement based on your overall experience with mobile internet service.<br />
  23. 23. Principles for Writing Good Questions<br />Use complete sentences to ask questions<br />Avoid vague quantifiers when more precise estimates can be obtained<br />The ease of using the same categories for many different kinds of questions has led to a tendency to use vague quantifiers<br />Avoid specificity that exceeds the respondent’s potential for having an accurate, ready-made answer<br />Use equal numbers of positive and negative categories for scalar questions<br />
  24. 24. Principles for Writing Good Questions<br />Offer a “no opinion” or “don’t know” option<br />Distinguish undecided from neutral by placement at the end of the scale<br />If an undecided category is offered to respondents, it makes a great deal of difference where that category is placed <br />Avoid bias from unequal comparisons<br />State both sides of attitude scales in the question stems<br />Example) Please indicate the extent to which you agree with each statement based on your overall experience with mobile internet service.<br />24<br />
  25. 25. Principles for Writing Good Questions<br />Eliminate check-all-that-apply question formats<br />Primary effect<br />Tendency to Select from among the first answer presented<br />Interview respondents are more likely to exhibit a recency effect.<br />Alternative<br />Rating scale for each item. Ranking question<br />Develop response categories that are mutually exclusive<br />Use cognitive design technique to improve recall<br />Using for the most important survey questions<br />Use for all questions would result in increased length of questionnaire, not to mention the greater tediousness of responding<br />Its use for the most important survey question<br />
  26. 26. Principles for Writing Good Questions<br />Provide appropriate time referents<br />Be sure each question is technically accurate<br />Choose question wording that allow essential comparisons to be made with previously collected data<br />Avoid asking respondents to say yes in order to mean no<br />Questions should not include double negatives<br />Avoid double-barreled questions<br />Soften the impact of potentially objectionable questions<br />Avoid asking respondents to make unnecessary calculations<br />26<br />
  27. 27. Summary<br />Principles for Writing Good Questions<br />Encouraging thoughtful answers<br />Cognitive design<br />Overcoming common wording problems<br />Be specific<br />Use simple words<br />Don’t be vague<br />Keep it short<br />Don’t talk down to respondents<br />Don’t be too specific<br />27<br />
  28. 28. 설문지 만들기[DON] Chapter 3. Constructing the questionnaire[COV] Chapter 3. The tools at hand<br />
  29. 29. Criteria for Design <br />Importance of questionnaire design<br />To reduce nonresponse<br />Respondent-friendly questionnaire design can improve response rate<br />To reduce or avoid measurement error<br />Achieving clear response to questions<br />To keep both the wording and visual appearance of questions simple<br />Voting Ballot Usability<br />
  30. 30. Alternative questionnaire formats and why a booklet is preferred<br />Unacceptable formats<br />Printing on both sides of sheets<br />Printing of pages in a landscape (horizontal) rather than portrait (vertical) orientation<br />Unusual folds<br />Unusual shapes<br />Acceptable formats for use with multi-page questionnaire<br />A booklet<br />Conventional legal size (8 ½ X 14”) paper<br />Full-sized pages<br />Two-column format allows more questions per pages<br />Print pages on one side only and staple them in the upper left corner<br />
  31. 31. Criteria For Ordering The Questions<br />Possible order effects between opinion questions<br />Norm of evenhandedness, a value-based effect<br />Phenomenon of adjusting answers to succeeding questions based on the previous answer was first noticed in interview surveys<br />Anchoring or cognitive-based order effect<br />Response to the first of opinion questions serves as an anchor for the second answer<br />Example<br />Cheating at this univ. (65%) cheating at univ. throughout U.S.A (60%)<br />Cheating at univ. throughout U.S.A (44%) cheating at this univ.(49%)<br />Addition or carryover effect<br />이전 질문에 대답에 의해 다음 질문에 대답이 영향을 받는 경우<br />Subtraction effect<br />Different kinds of order effects likely to occur on topically-related opinion questionsbut vague quantifier cannot be ignored<br />
  32. 32. Criteria For Ordering The Questions<br />Choose the first question carefully<br />Should be clearly apply to everyone<br />Should be easy<br />The first item in a questionnaire needs to be interesting<br />Connectedness between the respondent and survey purpose<br />Seldom begin a questionnaire with demographic questions<br />Demographic questions are usually at the end of pages<br />
  33. 33. Principle 3.6<br />Begin asking questions in the upper quadrant<br />The ideal place to start asking a question is near the upper left-hand corner of page<br />Principle 3.7<br />Use the largest and/or brightness symbols to identify starting point on each page<br />Page 107 Start Here<br />Principle 3.8<br />Identify the beginning of each succeeding question in a consistent way<br />Example<br />Reverse print numbers<br />Important navigational guides on the pages<br />Principles for constructing questionnaire pages<br />Principle 3.6 Example<br />Principle 3.8 Example<br />
  34. 34. Principle 3.9<br />Number questions consecutively and simply, from beginning to end<br />Example<br />P1, P2, H1, H2only relevant to agency division<br />1,2,3,4  easier for respondents to follow<br />Principle 3.10<br />Use a consistent figure/ground format to encourage the reading of all words<br />The use of reverse print in sentences or among answer categories should be avoided<br />Example<br />Principle 3.9 Example<br />There are printed<br />There are printed<br />Principles for constructing questionnaire pages<br />
  35. 35. Principle 3.16<br />Do not place instructions in a separate instruction book or in a separate section of the questionnaire<br />Decision of separating booklet has 4 consequence<br />Greatly increase the difficulty of completing the questionnarie<br />Suggestions<br />If instruction short after the question<br />If instruction are quite lengthy 페이지를 나누어 왼쪽은 question, 오른쪽은 instruction으로 물어봄<br />Principle 3.17<br />Use of lightly shaded colors as background fields on which to write all questions provides an effective navigational guide to respondents<br />Printed in 15-30% of full color or gray version<br />Principles for constructing questionnaire pages<br />Principle 3.16 Example<br />Principle 3.17 Example<br />
  36. 36. Principle 3.18 <br />When shaded background fields are used, identification of all answer spaces in white helps to reduce item nonresponse<br />Principle 3.19<br />List answer categories vertically instead of horizontally<br />Principles for constructing questionnaire pages<br />Principle 3.18 Example<br />
  37. 37. Principle 3.22<br />Vertical alignment of question subcomponents among consecutive questions eases the response task<br />Principle 3.23<br />Avoid double or triple baking of answer choices<br />Principle 3.22 Example<br /><ul><li>Principle 3.24
  38. 38. Maintain spacing between answer choices that is consistent with measurement intent</li></ul>Principles for constructing questionnaire pages<br />
  39. 39. Principle 3.25<br />Maintain consistency throughout a questionnaire in the direction scales are displayed<br /><ul><li>Principle 3.26
  40. 40. Use shorter lines to prevent word form being skipped.attentivelyon about 8 to 10 characters of text at
  41. 41. The preference for using two column formats on 8 ½ X11“ pages instead of entire page</li></ul>Principles for constructing questionnaire pages<br />Principle 3.25 Example<br />Principle 3.26 Example<br />
  42. 42. Step 3<br />Developing additional visual navigational guides, the aim of which is to <br />interrupt established navigation behavior and redirect respondents, <br />for example, through skip patterns<br />Principle 3.27<br />Major visual changes are essential for gaining compliance with skip patterns<br />오프라인 설문의 경우page 131<br />온라인 설문의 경우, 시스템적으로 해결 가능<br />Principles for constructing questionnaire pages<br />
  43. 43. Principle 3.28<br />Words and phrases that introduce important, but easy to miss, changes in respondent expectations should be visually emphasized consistently, but sparingly<br />Do not use reverse print, new color, different kinds of emphasis in one sentence<br />Use Italics or underline<br />Principles for constructing questionnaire pages<br />
  44. 44. Designing the questionnaire cover pages<br />Questionnaire cover designs<br />Improve response rate<br />The front cover<br />Black cover was more memorable and easierto retrieve form previous mail<br />Principles for designing separate cover pages<br />The questionnaire needs to be immediately distinguishablefrom all other questionnaires that a respondent might receive<br />Simple and neutral graphical design is often used<br />Detailed pictures should be avoided<br />The title should be short and simpleand, written to the respondent, not to the organization or agency sponsoring the survey<br />The name and address of the study sponsor are included<br />Respondent will know where to send the questionnaire <br />The questionnaire cover is viewed as an extension of the cover letter<br />
  45. 45. Designing the questionnaire cover pages<br />The back cover<br />Should be kept simple<br />Consisted of an invitation to make additional comments, a thank you, and plenty of white space<br />Questions are never included on the back page<br />Increase the chance of nonresponse<br />42<br />
  46. 46. Pretesting : Four sequential stages<br />Stage 1<br />Review by knowledgeable colleagues and analysts<br />To elicit suggestions based on experience with previous surveys and knowledge of study objectives<br />To finalize the substantive content of the questionnaire so the construction process can be undertaken<br />Stage 2<br />Interviews to evaluate cognitive and motivational qualities<br />Both Concurrent and retrospective technique use<br />Concurrent technique<br />Telling the interviewer everything they are thinking<br />Designed to produce information when the respondent is confused or can’t answer a question<br />Retrospective technique<br />지켜본 다음, 문제점에 대하여 설문이 끝난 후 진행<br />Especially useful in revealing navigational difficulties <br />
  47. 47. Pretesting : Four sequential stages<br />Stage 3<br />A small pilot study<br />Can estimate to be made for response rates, item nonresponse, and variable distribution<br />Stage 4<br />A finalcheck.<br />44<br />
  48. 48. The purpose of pretests<br />Testing questions<br />Variation<br />Testing items foran acceptable level of variationin the target population<br />Meaning<br />Whether the meaning intended by the investigator was shared by most respondents at the time<br />Task difficulty<br />Avoid asking respondents thing they know little about<br />Respondent interest and attention<br />Not everyone brings the same fervent interest to their topic that they do<br />
  49. 49. The purpose of pretests<br />Pretesting the questionnaire<br />“flow” and naturalness of the sections<br />A matter of intuitive judgment that it is hard to describe<br />The order of questions<br />Skip patterns<br />Must be pretested not only for the usual typographical error, but for the logic and format of skip patterns<br />Timing<br />Useful to ask interviewers to time each part of the questionnaire, section by section<br />Respondent interest and attention, overall<br />Respondent well-being<br />
  50. 50. Phases of Pretesting<br />Developmental Pretest 1<br />Most questions should beclosed<br />Three reason <br />Some boxes can be built to accommodate almost all case<br />Cost<br />같은 비용이라고 할 지라도 더 많은 사람들을 대상으로 실시할 수 있음<br />Validity<br />희망사항을 적으라고 하면 이것 저것 나오는데, intelligence에 대한 것은 나오지 않음.<br />Rough codes for open questions should be designed in advance<br />The first pretest interview should be less than twice the final expected length<br />Respondent should resemble the target population<br />A pretest N of25-75 is reasonable<br />
  51. 51. Phases of Pretesting<br />Evaluating pretests <br />Copious comments written in the margins of the schedule.<br />Oral debriefing<br />Written comments<br />Written questionnaires<br />“Did any of the questions seem to make R uncomfortable?”<br />“Did you have repeat any questions?”“Did R misinterpret any questions?”<br />“Which questions were the most difficult or awkward for you to read? Have you come to dislike any specific questions? Why?”<br />“Did any of the sections seem to drag?”<br />“Were there any sections in which you felt that the respondent would have liked the opportunity to say more?”<br />Field Observation<br />Coding of responses, and preparation of marginal frequencies<br />
  52. 52. Phases of Pretesting<br />Pretest II : the polishing pretest<br />“dress rehearsal” of the questionnaire as a whole<br />Ready for printer or final form on the computer<br />A time for cutting, trimming, splicing, rearranging, and filling in new skip patterns, formatting for clarity-polishing.<br />To evaluate the second pretest and prepare the final interview schedule.<br />Keep the barrier high to any new questions.<br />

×