How to Ask Survey Questions

886 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
886
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • How many of you participated in the first survey workshop?
    The previous workshop focused on the process from conception to delivery. Today’s workshop focuses on the design of the survey’s questions.
    Please keep in mind your own experiences with taking and designing surveys.
  • RV
  • RV
    5 minutes to take
    5 minutes to discuss
    Did you feel your opinions, attitudes, behavior were captured in these questions?
    How was the balance of types of questions, open, closed, rankings?
  • RV
    (open Checklist on LibWorks)
    Ask participants to recall some tasks and guidelines. Task to highlight: considering stakeholders and authority to act.
    Success depends on preparation.
  • RV
    A concrete question is precise and unambiguous. Use time periods that are related to the importance of the question.
    Periods of a year or more can be used for major life events while periods of a month or less should be used for questions that are less important. Asking people to remember relatively unimportant events over long periods of time leads to too much guessing.
    A survey is not a conversation. To get accurate information, survey questions rely on standard grammar, punctuation, and spelling. You should use words that maximize understanding for everyone in the survey. Use complete sentences, avoid abbreviations, avoid slang and colloquial expressions, and be careful of jargon and technical expressions.
    Negatives and double-negatives can be confusing.
    Purpose of loading is to encourage respondents to give a “true” response rather than one that is socially acceptable. Use loaded questions, if necessary, but be cautious as respondents may see through them, get annoyed, answer inaccurately or not at all.
    Avoid ‘two-edged’ questions which contain two ideas—look for “and” in question.

    Can you think of some examples of not concrete vs concrete, bias vs non-bias, etc.
    What do you think of the class and the instructor? (two-edged)
    What is your opinion of the library’s Access Services (jargon)
    Do you support the govt taking away personal choice by imposing govt mandated universal healthcare? (bias)
  • RV
    Use open ended questions when you want to have a range of responses, use close ended when you want to narrow down a range. Consider the purpose behind posing the question, the capabilities and willingness of the respondents, the level of analysis you will be able to apply to the results, and how you will report out the results. Again, preparation of the questions is very important in order to get the data you are looking for.
  • RV

    Likert Scale : Often used to assess a person's feelings about something.
    Multiple Choice : When there are a finite number of options, can allow respondents to choose more than one.
  • RV
    http://www.keene.edu/crc/forms/designingsurveysthatcount.pdf
  • RV
    As a respondent, you may feel frustrated with the options you are given.
  • cs

    Rina has outlined the advantages and disadvantages of closed end-ed questions. Understanding the pro’s and con’s of your question type will help you understand the what the data you collect can and cannot tell you. So first I’m going to look more in-depth about these question types and then Sarah will continue with choosing the type. Let’s take a closer look at these three types of closed ended questions.
  • CS

    Categorical Questions—you may also hear them referred to as Nominal Questions Questions for which the responses are assigned a number with no meaning. The number is used as a placeholder for that response.

    -Can add an other, free response category
    -Includes True/False and Yes/No questions

    A categorical variable (sometimes called a nominal variable) is one that has two or more categories, but there is no intrinsic ordering to the categories. For example, gender is a categorical variable having two categories (male and female) and there is no intrinsic ordering to the categories.  Hair color is also a categorical variable having a number of categories (blonde, brown, brunette, red, etc.) and again, there is no agreed way to order these from highest to lowest.  A purely categorical variable is one that simply allows you to assign categories but you cannot clearly order the variables.  If the variable has a clear ordering, then that variable would be an ordinal variable, as described below.
  • CS

    Categorical Questions—you may also hear them referred to as Nominal Questions Questions for which the responses are assigned a number with no meaning. The number is used as a placeholder for that response.

    -Can add an other, free response category
    -Includes True/False and Yes/No questions

    A categorical variable (sometimes called a nominal variable) is one that has two or more categories, but there is no intrinsic ordering to the categories. For example, gender is a categorical variable having two categories (male and female) and there is no intrinsic ordering to the categories.  Hair color is also a categorical variable having a number of categories (blonde, brown, brunette, red, etc.) and again, there is no agreed way to order these from highest to lowest.  A purely categorical variable is one that simply allows you to assign categories but you cannot clearly order the variables.  If the variable has a clear ordering, then that variable would be an ordinal variable, as described below.
  • CS
    Categories are not exhaustive. Survey likely to produce more responses in the other category than in the other three. Improve by adding more categories.

    Categories for the responses are not mutually exclusive. The choice professional can include all of the remaining categories, and registered nurse can include several of the other categories. To improve change to Which of the following apply to you? Answer yes or no for each category.
  • CS

    Nominal Questions Questions for which the responses are assigned a number with no meaning. The number is used as a placeholder for that response.

    -Can add an other, free response category -Includes True/False and Yes/No questions An ordinal variable is similar to a categorical variable.  The difference between the two is that there is a clear ordering of the variables.  For example, suppose you have a variable, economic status, with three categories (low, medium and high).  In addition to being able to classify people into these three categories, you can order the categories as low, medium and high. Now consider a variable like educational experience (with values such as elementary school graduate, high school graduate, some college and college graduate). These also can be ordered as elementary school, high school, some college, and college graduate.  Even though we can order these from lowest to highest, the spacing between the values may not be the same across the levels of the variables.  Say we assign scores 1, 2, 3 and 4 to these four levels of educational experience and we compare the difference in education between categories one and two with the difference in educational experience between categories two and three, or the difference between categories three and four. The difference between categories one and two (elementary and high school) is probably much bigger than the difference between categories two and three (high school and some college).  In this example, we can order the people in level of educational experience but the size of the difference between categories is inconsistent (because the spacing between categories one and two is bigger than categories two and three).  If these categories were equally spaced, then the variable would be an interval variable, like a Likert scale.
  • CS
    Examples include frequency and intensity Likert scales

    When crafting a Likert scale, try to keep the values of each interval roughly the same. Likert scale questions are similar to other ordinal questions, except that the intervals between the answers are equally spaced.  For example, suppose you have a variable such as annual income that is measured in dollars, and we have three people who make $10,000, $15,000 and $20,000. The second person makes $5,000 more than the first person and $5,000 less than the third person, and the size of these intervals  is the same.  If there were two other people who make $90,000 and $95,000, the size of that interval between these two people is also the same ($5,000).
  • CS
    Looks a lot like a likert scale question, but it can be confusing to the user. Because only one ranking can be selected per category of ranking. Ex. Only one of the responses can be a 5, one a 4 and one a 3…etc.
    Difference between answers not a meaningful interval

  • CS
    Looks a lot like a likert scale question, but it can be confusing to the user. Because only one ranking can be selected per category of ranking. Ex. Only one of the responses can be a 5, one a 4 and one a 3…etc.
    Difference between answers not a meaningful interval

  • CS
  • cgs

    Likert Scale : Often used to assess a person's feelings about something.
    Multiple Choice : When there are a finite number of options, can allow respondents to choose more than one.
  • cgs

    Likert Scale : Often used to assess a person's feelings about something.
    Multiple Choice : When there are a finite number of options, can allow respondents to choose more than one.
  • SB
    As mentioned in previous class, there is a reason or goal for each survey. It could be a large question or concept. For example, What services do graduate students expect from their library? Once you have the survey goal or question, you need to think about the actions that will result from the survey. Will you stop doing something? Start doing something? Purchase something? Include different content in a class?

    Work backwards from goals to data to questions
  • SB
  • SB
    Multiple surveys, 3-5 (political, library), different groups do the work


    Indentify some questions on each survey for the group to pay review for the questions in the exercise above

    Green Attitudes: 5-8, 10, 29
    Vogue Knitting: #5-6, 9, 12, 28, 29, 47
    Great Streets: #20, 26-28, 35
  • SB
  • SB
    Turn into handout!!!
  • SB
  • SB
    Be sure to use a meaningful scale, not-biased
  • SB
    Not too many in a big block
    Odd number provides neutral, even does not
  • SB
  • SB
    Each goal will require more than one question to be fully answered. That’s ok. Please select one to work on.
  • SB
    Ask class for things they hate about surveys we can avoid?
  • RV
  • How to Ask Survey Questions

    1. 1. + How to Ask Survey Questions Workshop on Survey Design
    2. 2. + Learning Objectives  After this class you will be able to: 1. Construct questions that are clear to your intended audience 2. Construct questions requiring ordered responses or scales 3. Choose among question types 4. Check that questions will provide data that meets your goal
    3. 3. + Exercise 1: Survey Analysis Look over and take the Green Attitudes survey  What are your reactions to the survey?  What did you like about the survey? Dislike?  What do you think about the content and length of the survey?
    4. 4. + Overview of Previous Workshop  Concepts  Checklist of Survey Tasks  Overview of the survey process  Guidelines for Asking Questions  Best practices for question wording
    5. 5. + Review: Guidelines for Asking Questions  Ask purposeful, concrete questions that use conventional language  Avoid bias in words and phrasing  Avoid two-edged questions and negative questions  Use shorter questions  Choosing between Open and Closed Questions
    6. 6. + Review: Selecting Open and Closed Ended Questions  Copy over Use Open Use Closed Purpose Respondents’ own words essential Want data that are rated or ranked and can order the ratings in advance Respondents Are capable and willing to provide answers in their own words Are unwilling or unable to express themselves while being surveyed Questions Prefer open because choices are unknown Using a pre-specified set of response choices Analysis Have ability to analyze variety of comments Prefer to count the number of choices Reporting Provide individual or grouped verbal responses Reports statistical data
    7. 7. + Types of Closed Questions  Categorical: The number next to each response has no meaning except as a placeholder for that response.  Multiple Choice: Select one or more answers.  Ordinal: Assigns a meaningful number to responses, puts things in order.  Likert Scale: To assess a person's feelings about something.  Ranking: Rank things in relation to other things.  Numerical: For real numbers, like age, number of months.
    8. 8. + Advantages of Closed-Ended Questions  Easy to code, enter, and analyze  Easy to present  Quick turnaround  Enhanced reliability  Less researcher bias  High degree of anonymity
    9. 9. + Disadvantages of Closed Ended Questions  Harder to develop questions and response categories  May force invalid responses  Less depth and substance  Respondents unable to explain, qualify, or clarify answer During the past month, have you felt depressed? 0 = No 1 = Yes, once in a while 2 = Yes, some of the time 3 = Yes, most of the time 4 = Yes, all of the time
    10. 10. + 3 Types of Closed Questions 1. Categorical 2. Ordinal  Likert Scale  Ranking 3. Numerical
    11. 11. + Categorical Questions, ex. 1-2  Gender  Male  Female  Transgender  Color of eyes  Blue  Black  Brown  Hazel  Ask respondents to tell which category they fit into  Response choices should be mutually exclusive  Response choices should be inclusive and exhaustive  Use groups that make sense in the survey and will be useful in reporting results  Can allow respondents to choose more than one answer
    12. 12. + Categorical Questions, ex. 3  Ask respondents to tell which category they fit into  Response choices should be mutually exclusive  Response choices should be inclusive and exhaustive  Use groups that make sense in the survey and will be useful in reporting results  Can allow respondents to choose more than one answer  4. My child/children attend(s)  Public school  Non-local public school  Charter school  Private school  Does not attend school yet  Graduated  Other (please specify) __________________
    13. 13. + Exercise 2: Improve Poor Categorical Questions 1. Which one of the following best describes your primary expertise?  Landlord-tenant problems  Consumer problems  Traffic cases  Other (specify) 2. Which of the following best describes you?  Professional  Registered nurse  Nurse practitioner  Administrator  Nurse midwife Professional RN Practitioner
    14. 14. + Ordinal Questions, ex. 1-2  How satisfied are you with how long you can check out books? 1. Very dissatisfied 2. Dissatisfied 3. Satisfied 4. Very Satisfied  What is the highest level of education that you have completed? 1. Elementary school 2. High school graduate 3. Some college 4. College graduate  Use a meaningful scale  Balance all responses  Use don’t know or neutral carefully  Use 4- to 7-point rating scales, with negative end first  Keep uncluttered and easy to complete
    15. 15. + Ordinal Questions: Likert Scales  Considering your reading habits, during the past year how often did you read the following newspapers, journals, and magazines? Circle one number for each periodical. Periodical Never (1) Rarely (2) Sometimes (3) Frequently (4) Always (5) New York Times 1 2 3 4 5 Wall Street Journal 1 2 3 4 5 Cosmopolitan 1 2 3 4 5 New England Journal of Medicine 1 2 3 4 5 Sports Illustrated 1 2 3 4 5
    16. 16. + Ordinal Questions: Ranking ex. 1  Rank items against each other  Each item is assigned a number  Don’t make list too long  Can be very confusing, use sparingly Based upon what you have seen, heard, and experienced, please rank the following brands according to their reliability. Place a "1" next to the brand that is most reliable, a "2" next to the brand that is next most reliable, and so on. Remember, no two cars can have the same ranking . __ Honda __ Toyota __ Mazda __ Ford
    17. 17. + Ordinal Questions: Ranking ex. 2  Rank items against each other  Each item is assigned a number  Don’t make list too long  Can be very confusing, use sparingly
    18. 18. + Exercise 3: Improve Poor Ordinal Questions 1. How satisfied were you with your hotel stay?  Completely Satisfied, Very Satisfied, Pretty Satisfied, Satisfied, A Little Satisfied, Slightly Satisfied, Unsatisfied, Don’t Know 2. Please rate your instructor on the following criteria
    19. 19. + Numerical Questions  Numerical: For real numbers, like age, number of months. 1. How old are you? 2. In what year were you born? 3. what is your annual income?
    20. 20. + Types of Closed Questions  Categorical: The number next to each response has no meaning except as a placeholder for that response.  Multiple Choice: Select one or more answers.  Ordinal: Assigns a meaningful number to responses, puts things in order.  Likert Scale: To assess a person's feelings about something.  Ranking: Rank things in relation to other things.  Numerical: For real numbers, like age, number of months.
    21. 21. + Survey Goals Revisited  What question or questions are you trying to answer?  What information do you need to decide upon an answer?  What actions will you take when you have the survey’s results?  Start or stop doing something?  Make a purchase?  Select content for a class?  Who is your audience?
    22. 22. + Every Question Should Have a Purpose  Keep your survey as short as possible  Every question should provide data you will use to make a decision  Don’t automatically include questions you’ve seen before  Don’t just use questions that are about your topic without thinking about the resulting actions possible from the data  Be sure that every answer will cause you to do something differently  Don’t ask questions just to satisfy curiosity
    23. 23. + Exercise 4: Survey Audience and Goals  Look over your group’s survey  Green Attitudes : 5-8, 10, 29  Great Streets : 20, 26-28, 35  Vogue Knitting: 5-6, 9, 12, 28, 29, 47  What were the goals of the survey creator?  Who is the audience of the survey?  Look carefully at each question  Will the data gathered meet the goals of the survey?  Will the answers to any individual question make a choice clearer?
    24. 24. + Kinds of Information  Copy over Description Example Knowledge What people know; how well they understand something True or False:The most effective weight loss plan includes exercise. Belief What people think is true; an opinion Do you think that lower food prices would increase consumption? Attitude How people feel about something; a preference Do you favor or oppose a smoking ban in your county? Behavior What people do; can be mental or physical Have you ever attended a Library workshop? Attributes What people are; what people have Demographics: What year do you plan to graduate?
    25. 25. + When to Use Categorical Questions  Can be used to find all types of information, but answer choices are limited  To ask many questions in a short time period  When there are a finite number of options  Be careful in analysis  Cannot compare one item to another  Can create a frequency distribution  If only one answer allowed, you don’t have data about the other answers
    26. 26. + When to Use Ordinal Questions: Ranking  Best used to assess attitudes and knowledge  To rate things in relation to other things  When there are a finite number of options  Be careful in analysis  There is not a meaningful interval between choices  Can create a frequency distribution or cumulative frequency distribution  Cannot average, add, subtract data
    27. 27. + When to Use Ordinal Questions: Likert-Scale  Best used to assess beliefs and attitudes  Sample scales  Very dissatisfied, somewhat dissatisfied, somewhat satisfied, very satisfied  Strongly disagree, disagree somewhat, uncertain, agree somewhat, strongly agree  Be careful in analysis  Be sure your intervals are meaningful and equal  Can average, add, subtract, and create frequency distributions
    28. 28. + When to Use Numerical Questions  Best used for attributes (demographics)  For precise real numbers, not ranges  Can analyze using most methods  Multiply, divide, average, add, subtract, and frequency distributions  If your survey tool doesn’t support this data type, you can approximate with an open-ended question  You will have to manually analyze data
    29. 29. + Exercise 5: Matching Question Types to Goals 1. Name one action you might you take with each goal’s results. 2. What type of data do you need to meet each goal? 3. What question type will help you get that data? 4. Write a sample question for each goal.  Goal 1: Determine if patrons know what services we offer at the Reference Desk.  Goal 2: Determine if people are satisfied with the wait times at the Circulation Desk.  Goals 3: Determine if the respondents to your survey are representative of the whole campus.
    30. 30. + Don’t Forget…  Ask enough demographic questions to ensure that your intended audience is represented  Ask as few questions as possible  Don’t ask too many open-ended questions  Will lower response rate  One is usually enough  Don’t have too many Likert-scale questions in a large block
    31. 31. + Learning Objectives Revisited  After this class you will be able to: 1. Construct questions that are clear to your intended audience 2. Construct questions requiring ordered responses or scales 3. Choose among question types 4. Check that questions will provide data that meets your goal
    32. 32. + Templates

    ×