7 measurement & questionnaires design (Dr. Mai,2014)
•
7 likes•7,353 views
This document provides information on designing measurement questionnaires. It discusses different types of scales including nominal, ordinal, interval and ratio scales. Examples are given for different scale types like Likert scales, semantic differential scales, ranking scales. Guidelines are outlined for writing good questionnaire questions, including sequencing questions from easy to difficult, avoiding bias, and pilot testing. Tips are also given for constructing questionnaires like keeping them short, starting with an introduction, and exhaustively listing answer choices. The document aims to help with strategically designing measurement instruments and questionnaires.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Similar to 7 measurement & questionnaires design (Dr. Mai,2014)
Similar to 7 measurement & questionnaires design (Dr. Mai,2014) (20)
More from Phong Đá
Recently uploaded
Recently uploaded (20)
7 measurement & questionnaires design (Dr. Mai,2014)
- 1. 7. Measurement & Questionnaires Design Dr. Nguyen Quynh Mai
- 2. Measurement Give examples: • What are some things that are easy to measure? • What are some things that are difficult to measure?
- 4. Measurement • Quantitative measurement involves assigning numbers to attributes or constructs in order to create variables so that we can better understand the relationships among variables • With qualitative inquiry involves assigning words to attributes or concepts in order to draw out themes related to human perceptions and meaning
- 5. Measurement • If our studies do not allow us to measure variation in the dependent variable (Y) as related to variation in our X variables, then we cannot do any scientific testing. 1. We measure whether certain variables are meaningful – individually significant. 2. We measure the variation in our variables. 3. We also measure the significance and explanatory power of our models and the relationships between variables. 4. If it can be quantified, then you should do so.
- 6. Qualities of Variables • Exhaustive -- Should include all possible answerable responses. (Schooling: No Schooling, Elementary, Middle, HS, College) • Mutually exclusive -- No respondent should be able to have two attributes simultaneously (e.g. Female Male ).
- 7. What Is Level of Measurement? The relationship of the values that are assigned to the attributes for a variable 1 2 3 Relationship Values Attributes Variable Low Medium High Development
- 9. 9 Levels of Measurement Ordinal interval Ratio Nominal Classification Order Classification Order Classification Distance Natural Origin Order Classification Distance
- 10. 10 Nominal Scales • Nominal or categorical data is data that comprises of categories that cannot be rank ordered – each category is just different • Mutually exclusive and collectively exhaustive categories • Exhibits the classification characteristic only. Nominal data reflect qualitative differences rather than quantitative ones. • Variables that have only two responses i.e. Yes or No, are known as dichotomies • Examples: What is your gender? (please tick) Male Female Did you enjoy the film? (please tick) Yes No
- 11. 11 Ordinal Scales • Characteristics of nominal scale plus an indication of order • Implies statement of greater than and less than • Ex: Doneness of meat (well, medium well, medium rare, rare)
- 12. • Both interval and ratio data are examples of scale data. • Scale data: data is in numeric format (£50, £100, £150) data that can be measured on a continuous scale the distance between each can be observed and as a result measured the data can be placed in rank order. Interval and ratio data
- 13. 13 Interval Scales • Interval data measured on a continuous scale and has no true zero point • Characteristics of nominal and ordinal scales plus the concept of equality of interval. • Equal distance exists between numbers • Examples: • Time – moves along a continuous measure or seconds, minutes and so on and is without a zero point of time. • Temperature – moves along a continuous measure of degrees and is without a true zero.
- 14. 14 Ratio Scales • Characteristics of previous scales plus an absolute zero point • Examples – Weight – Height – Number of children – Age
- 17. Evaluating Measurement Tools Criteria Validity Practicality Reliability
- 21. 21 Increasing Construct Validity New measure of trust Known measure of trust Empathy Credibility
- 22. 22 Judging Criterion Validity Relevance Freedom from bias Reliability Availability Criterion
- 24. Reliability • Reliability is defined as the extent to which a questionnaire, test, observation or any measurement procedure produces the same results on repeated trials • Equivalence refers to the amount of agreement between two or more instruments that are administered at nearly the same point in time. Equivalence is measured through a parallel forms procedure in which one administers alternative forms of the same measure to either the same group or different group of respondents. • Stability occurs when the same or similar scores are obtained with repeated testing with the same group of respondents. Stability is assessed through a test-retest procedure that involves administering the same measurement instrument to the same individuals under the same conditions after some period of time. • Internal consistency concerns the extent to which items on the test or instrument are measuring the same thing. If the individual items are highly correlated with each other you can be highly confident in the reliability of the entire scale (Miler, Western International University) 24
- 25. Equivalence test • Reliability is established by using similar/alternate forms (Forms A & B) that measure the same trait/knowledge. • Two forms are created by splitting the questions on the test randomly before administration of the forms. One group of students gets form "A" first, then "B". Another group takes the "B" form of the test first, then the "A" version. The scores on both are then correlated, producing a correlation or reliability coefficient. 25
- 26. Stability test: Test-Retest • If you use a test with a student in the morning and then administer it again in the afternoon, would you expect about the same results? • Which would have the highest test-retest reliability coefficient (correlation)? – retesting a youngster on an achievement test within the same week – retesting in different years 26
- 27. Internal consistency • The internal consistency reliability of survey instruments is a measure of reliability of different survey items intended to measure the same characteristic. • For example, there are 5 different questions (items) related to anxiety level. Each question implies a response with 5 possible values on a Likert scale (1 – 5). Responses from a group of respondents have been obtained. In reality, answers to different questions vary for each particular respondent, although the items are intended to measure the same aspect or quantity. The stronger the correlation, the greater the internal consistency reliability of this survey instrument. • In statistic, they use Cronbach alpha to measure reliability 27
- 30. 30 Nature of Attitudes: example of measurement Cognitive I think oatmeal is healthier than corn flakes for breakfast. Affective/ attitude Behavioral I hate corn flakes. I intend to eat more oatmeal for breakfast.
- 31. 31 Improving Predictability Reference groups Multiple measures Factors Strong Specific Basis Direct
- 32. 32 Selecting a Measurement Scale Research objectives Response types Data properties Number of dimensions Forced or unforced choices Balanced or unbalanced Rater errors Number of scale points
- 33. 33 Response Types Rating scale Ranking scale Categorization Sorting Number of Dimensions Unidimensional Multi-dimensional
- 34. Balanced or Unbalanced Very bad Bad Neither good nor bad Good Very good Poor Fair Good Very good Excellent How good an actress is Angelina Jolie?
- 35. Forced or Unforced Choices Very bad Bad Neither good nor bad Good Very good Very bad Bad Neither good nor bad Good Very good No opinion Don’t know How good an actress is Angelina Jolie?
- 36. Number of Scale Points Very bad Bad Neither good nor bad Good Very good Very bad Somewhat bad A little bad Neither good nor bad A little good Somewhat good Very good How good an actress is Angelina Jolie?
- 37. Simple Category Scale I plan to purchase a Samsung laptop in the 12 months. Yes No
- 38. Multiple-Choice, Single-Response Scale What newspaper do you read most often for financial news? East City Gazette West City Tribune Regional newspaper National newspaper Other (specify:_____________)
- 39. Multiple-Choice, Multiple-Response Scale What sources did you use when designing your new home? Please check all that apply. Online planning services Magazines Independent contractor/builder Designer Architect Other (specify:_____________)
- 40. Likert Scale The Internet is superior to traditional libraries for comprehensive searches. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree
- 42. SD Scale for Analyzing Actor Candidates
- 43. Numerical Scale
- 44. Multiple Rating List Scales
- 45. Ranking Scales • Paired-comparison scale • Forced ranking scale • Comparative scale
- 49. Overall Flowchart for Instrument Design
- 50. Flowchart for Instrument Design: Phase 1
- 51. Strategic Concerns in Instrument Design What type of scale is needed? What communication approach will be used? Should the questions be structured? Should the questioning be disguised?
- 52. Flowchart for Instrument Design: Phase 2
- 53. Question Categories and Structure Administrative Classification Target
- 54. Question Content Should this question be asked? Is the question of proper scope and coverage? Can the participant adequately answer this question as asked? Will the participant willingly answer this question as asked?
- 56. Response Strategy Factors Objectives of the study Participant’s level of information Degree to which participants have thought through topic Ease and clarity with which participant communicates Participant’s motivation to share
- 57. Free-Response Strategy What factors influenced your enrollment in Metro U? ____________________________________________ ____________________________________________
- 58. Dichotomous Response Strategy Did you attend the “A Day at College” program at IU? Yes No
- 59. Multiple Choice Response Strategy Which one of the following factors was most influential in your decision to attend Metro U? Good academic standing Specific program of study desired Enjoyable campus life Many friends from home High quality of faculty
- 60. Checklist Response Strategy Which of the following factors influenced your decision to enroll in Metro U? (Check all that apply.) Tuition cost Specific program of study desired Parents’ preferences Opinion of brother or sister Many friends from home attend High quality of faculty
- 61. Rating Response Strategy Strongly influential Somewhat influential Not at all influential Good academic reputation Enjoyable campus life Many friends High quality faculty Semester calendar
- 62. Ranking Please rank-order your top three factors from the following list based on their influence in encouraging you to apply to Metro U. Use 1 to indicate the most encouraging factor, 2 the next most encouraging factor, etc. _____ Opportunity to play collegiate sports _____ Closeness to home _____ Enjoyable campus life _____ Good academic reputation _____ High quality of faculty
- 63. Summary of Scale Types Type Restrictions Scale Items Scale points Data Type Rating Scales Simple Category Scale Needs mutually exclusive choices One or more 2 Nominal Multiple Choice Single-Response Scale Needs mutually exclusive choices; may use exhaustive list or ‘other’ many 2 Nominal Multiple Choice Multiple-Response Scale (checklist) Needs mutually exclusive choices; needs exhaustive list or ‘other’ many 2 Nominal Likert Scale Needs definitive positive or negative statements with which to agree/disagree One or more 5 Ordinal Likert-type Scale Needs definitive positive or negative statements with which to agree/disagree One or more 7 or 9 Ordinal
- 64. Summary of Scale Types Type Restrictions Scale Items Scale points Data Type Semantic Differential Scale Needs words that are opposites to anchor the graphic space. One or more 7 Ordinal Numerical Scale Needs concepts with standardized or defined meanings; needs numbers anchor the end-points or points along the scale; score is a measurement of graphical space from one anchor. One or many 3-10 Ordinal or Interval Multiple Rating List Scale Needs words that are opposites to anchor the end-points on the verbal scale Up to 10 5-7 Ordinal Fixed Sum Scale Participant needs ability to calculate total to some fixed number, often 100. Two or more none Interval or Ratio Stapel Scale Needs verbal labels that are operationally defined or standard. One or more 10 Ordinal or Interval
- 65. Summary of Scale Types Type Restrictions Scale Items Scale points Data Type Ranking Scales Graphic Rating Scale Needs visual images that can be interpreted as positive or negative anchors; score is a measurement of graphical space from one anchor. One or more none Ordinal (Interval, or Ratio) Paired Comparison Scale Number is controlled by participant’s stamina and interest. Up to 10 2 Ordinal Forced Ranking Scale Needs mutually exclusive choices. Up to 10 many Ordinal or Interval Comparative Scale Can use verbal or graphical scale. Up to 10 Ordinal
- 66. Internet Survey Scale Options
- 67. Internet Survey Scale Options
- 68. Internet Survey Scale Options
- 69. Flowchart for Instrument Design: Phase 3
- 70. Guidelines for Question Sequencing Interesting topics early Classification questions later Sensitive questions later Simple items early Transition between topics Reference changes limited
- 71. Components of Questionnaires Introduction Transition Instructions for …. Conclusion a. Terminating b. Participant discontinue c. Skip d. Disposition instruction
- 72. Tips for a good questionnaires 1. Keep it short and simple 2. Start with a brief introduction explaining the purpose of the questionnaire 3. Ask yourself what you will do with the information from each question 4. Put easier questions first 5. Ask just one thing in a question 6. Leave difficult or sensitive questions towards the end 7. Try to be exhaustive when offering answer choices 8. Avoid bias in language
- 73. Tips for a good questionnaires (Cont) 9. Avoid jargon (special words that are used by a particular profession group and are difficult for others to understand) 10. Do not use "emotional language" or leading questions 11. Present Disagree/Agree choices in that order (disagree to agree); Same with Excellent to Poor, and Positive to Negative 12. User higher numbers to signify a more positive answer in rating scales 13. Be consistent with your layout 14. Separate related questions 15. Have Don’t Know or Not Applicable as an option for most questions
- 74. Tips for a good questionnaires (Cont) 16. Have Other or None for questions with a list of options 17. Include Other Comments at the end 18. Keep the number of open-ended questions to a minimum 19. Make sure questions are relevance with scale 20. Pilot test the questionnaire