Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Principles of Survey Research (questionStar)

806 views

Published on

An introductory course into survey and marketing research.

CONTENTS
1. Introduction
1.1 Market Research and Survey
1.2 Types of Market Research

2. Survey: Measurement and Scaling
2.1 Introduction
2.2 Comparative Scales
2.3 Non-Comparative Scales
2.4 Multi-item Scales
2.5 Reliability and Validity

3. Questionnaire
3.1 Asking Questions
3.2 Overcoming Inability to Answer
3.3 Overcoming Unwillingness to Answer
3.4 Increasing Willingness of Respondents
3.5 Determining the Order of Questions
3.6 What’s Next?

4. Sampling
4.1 Non-probability Sampling
4.2 Probability Sampling
4.3 Choosing Non-probability vs. Probability Sampling
4.4 Sample Size

5. Data Analysis: A Concise Overview of Statistical Techniques
5.1 Descriptive Statistics: Some popular Displays of Data
5.1.1 Organizing Qualitative Data
5.1.2 Organizing Quantitative Data
5.1.3 Summarizing Data Numerically
5.1.4 Cross-Tabulations
5.2 Inferential Statistics: Can the Results Be Generalized to Population?
5.2.1 Hypotheses Testing
5.2.2 Strength of a Relationship in Cross-Tabulation
5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables

6. Advanced Techniques of Market Analysis: A Brief Overview of Some Useful Concepts
6.1 Conjoint-Analysis
6.2 Market Simulations
6.3 Segmentation
6.4 Perceptual Positioning Maps

7. Reporting Results


Published in: Marketing
  • Hello! I can recommend a site that has helped me. It's called ⇒ www.HelpWriting.net ⇐ They helped me for writing my quality research paper.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • You might get some help from ⇒ www.HelpWriting.net ⇐ Success and best regards!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • 7 Second Daily Ritual Melts Stubborn Fat While You Sleep ●●● http://tinyurl.com/y33eq8tt
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • This simple 7 second daily ritual was tested with a group of volunteers. In total 62 people, all suffering from varying levels of obesity. After ninety days the results were more incredible than any of them could have ever imagined… On average they lost 54 pounds of fat... They experienced increased energy… Better mood… Several of them could even see their abs for the first time ever… This simple 7 second ritual had actually worked on everyone! Regardless of how much they had to lose… Their metabolism… Their genetics… or their age. Click Here To Learn More ➽➽ http://tinyurl.com/y46t9ja3
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I shared this simple 7 second ritual with one of my clients a couple of month ago. She had been struggling with her weight pretty much all her life. And seemed to be stuck in a perpetual limbo of weight loss, weight gain, weight loss, weight gain... But when she started doing THIS 7 second daily ritual things rapidly changed... She had fast fat loss from her hips, thighs face and arms.While enjoying an increase in energy and zest for life. She also had a youthful glow to her skin that she hadn't seen in years. >> Try This Simple 7 Second Daily Ritual For Yourself Here ☺☺☺ https://bit.ly/3fJGS4i
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Principles of Survey Research (questionStar)

  1. 1. Paul Marx | Principles of survey research Principles of Survey Research 1 introductory course
  2. 2. Paul Marx | Principles of survey research Contents 1. Introduction 1.1 Market Research and Survey 1.2 Types of Market Research 2. Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 3. Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 4. Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 5. Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 6. Advanced Techniques of Market Analysis: A Brief Overview of Some Useful Concepts 6.1 Conjoint-Analysis 6.2 Market Simulations 6.3 Segmentation 6.4 Perceptual Positioning Maps 7. Reporting Results 2
  3. 3. Paul Marx | Principles of survey research 1.Introduction 1.1 Market Research and Survey 1.2 Types of Market Research 3
  4. 4. Paul Marx | Principles of survey research 1.Introduction 1.1 Market Research and Survey 1.2 Types of Market Research 4
  5. 5. Paul Marx | Principles of survey research What is Research? Research is the systematic investigation into and study of materials and sources in order to establish facts and reach new conclusions. (Oxford Dictionaries) 5 Research is the searching for and gathering of information and ideas in response to a specific question. (Unknown author)
  6. 6. Paul Marx | Principles of survey research Survey Research 6 Survey - The most popular technique for gathering primary data in which a researcher interacts with people to obtain facts, opinions, and attitudes.
  7. 7. Paul Marx | Principles of survey research The Essence of Market Research 7 Researcher Decision Maker Obvious Measurable Symptoms Real Business/Decision Problems Unhappy Customers Decreased Market Share Loss of Sales Low Traffic Low-Quality Products Poor Image Marginal Performance of Sales Force Inappropriate Delivery System Unethical Treatment of Customers Decision Problem Definition
  8. 8. Paul Marx | Principles of survey research Who Why Sociology and Political Science Public opinion research, identification of population's attitudes towards socially important phenomena, events, and facts… Psychology Personality tests, intelligence tests, identification of individual strengths and weaknesses psychological stability, cognitive disorders, social influence… Human Resources Measurement of employee satisfaction, loyalty, potential, personality traits and leadership skills, productivity and quality of work, professional fit, resistance to stress, social intelligence, work-life balance… Marketing Market and consumer research, measurement of perception of image, preferences, attitudes, satisfaction with product and/or service, loyalty, willingness to pay; segmentation, positioning, new product development, evaluation of market potentials, pricing and price setting, advertising tests, ease of web-site navigation, user feedback, willingness to recommend... Science (in general) Study of relationships between two or more variables, factors, phenomena; development of scales and survey techniques for practical use… Education Knowledge tests (quizzes, exams), evaluation of students and/or teachers… … … Practical Application of Surveys 8
  9. 9. Paul Marx | Principles of survey research Market Research Process Define the Research problem Develop the research plan Collect data Analyze data Report findings 9 ⁻ identify and clarify information needs ⁻ define research problem and questions ⁻ specify research objectives ⁻ confirm information value If a problem is vaguely defined, the results can have little bearing on the key issues Decide on ⁻ budget ⁻ data sources ⁻ research approaches ⁻ sampling plan ⁻ contact methods ⁻ methods of data analysis The plan needs to be decided upfront but flexible enough to incorporate changes or iterations ⁻ collect data according to the plan or ⁻ employ an external firm This phase is the most costly and the most liable to error Analyze data ⁻ statistically or ⁻ subjectively and infer answers and implications Type of data analysis depends on type of research - Formulate conclusions and implications from data analysis - prepare finalized research report Overall conclusions to be presented rather than overwhelming statistical methodologies
  10. 10. Paul Marx | Principles of survey research When NOT to Conduct Market Research Occasion Comments Vague objectives When managers cannot agree on what they need to know to make a decision. Market research cannot be helpful unless it is probing a particular issue. Closed mindset When decision has already been made. Research is used only as a rubber stamp of a preconceived idea. Late timing When research results come too late to influence the decision. Poor timing If a product is in a “decline” phase there is little point in researching new product varieties. Lack of resources If quantitative research is needed, it is not worth doing unless a statistically significant sample can be used. When funds are insufficient to implement any decisions resulting from the research. Costs outweigh benefits The expected value of information should outweigh the costs of gathering an analyzing the data.. Results not actionable Where, e.g., psychographic data is used which will not help he company form firm decisions. 10
  11. 11. Paul Marx | Principles of survey research 1.Introduction 1.1 Market Research and Survey 1.2 Types of Market Research 11
  12. 12. Paul Marx | Principles of survey research Types of Market Research 12 By Objectives • Exploratory (a.k.a. diagnostic) • Descriptive • Causal (a.k.a. predictive, experimental) By Data Source • Primary • Secondary By Methodology • Qualitative • Quantitative
  13. 13. Paul Marx | Principles of survey research Market Research by Objectives •Explaining data or actions to help define the problem •What was the impact on sales after change in the package design? •Do promotions at POS influence brand awareness? Exploratory a.k.a. diagnostic •Gathering and presenting factual statements: who, what, when, where, how •What is historic sales trend in the industry? •What are consumer attitudes toward our product? Descriptive •Probing cause-and-effect relationships; “What if?” •Specification of how to use the research to predict •the results of planned marketing decisions •Does level of advertising determine level of sales? Causal a.k.a. predictive, experimental 13 Survey of a small sample, focus groups, depth interviews,,… Survey of a large representative sample, observation, … Experiments, A&B tests, consumer panels, … Uncertainty influences the type of research UncertainCertain
  14. 14. Paul Marx | Principles of survey research Market Research by Data Source 14 • Original research to collect new raw data for a specific reason. This data is then analyzed and may be published by the researcher. Primary • Research data that has been previously collected, analyzed and published in the form of books, articles, etc. Secondary Survey, Interviews, observation, experiments, … Literature review, library, web, database, archive,…
  15. 15. Paul Marx | Principles of survey research Market Research by Methodology 15 • Involves collecting and measuring data • Often requires large data sets. For example, large number of people. • Uses statistical methods to analyze data • Aims to achieve objective/scientific view of the subject Quantitative • Involves understanding human behavior and the reasons behind it • Focus is on individuals and small groups • Objectivity is not the goal, the aim is to understand one point of view, not all points of view. • Usually not representative Qualitative Survey of a large representative sample, observation, … Survey of a small sample, focus groups, depth interviews,,…
  16. 16. Paul Marx | Principles of survey research 16 APPARENT TRUTH Literature Review InterviewSurvey Triangulation Robson (1998), Visocky & Visocky (2009)
  17. 17. Paul Marx | Principles of survey research 17
  18. 18. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 18
  19. 19. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 19
  20. 20. Paul Marx | Principles of survey research Measurement Measurement – assigning numbers or other symbols to characteristics of objects according to certain pre- specified rule - one-to-one correspondence between the numbers and characteristics being measured - the rules for assigning numbers should be standardized and applied uniformly - rules must not change over objects or time 20
  21. 21. Paul Marx | Principles of survey research Scaling Scaling – involves creating a continuum upon which measured objects are located. 21 Extremely favorable Extremely unfavorable
  22. 22. Paul Marx | Principles of survey research Primary Scales of Measurement 22 • numbers serve as labels for identifying and classifying objects • not continuosNominal • numbers indicate the relative positions of objects • but not the magnitude of difference between themOrdinal • differences between objects can be compared • zero point is arbitraryInterval • zero point is fixed • ratios of scale values can be computedRatio a.k.a. metric or 1 2 1 2 1 2 NOT 3 1 2 1 2 3 My preference as a snack food moreless 0 25 50 75 100 Weight(kg)
  23. 23. Paul Marx | Principles of survey research Primary Scales of Measurement Scale Basic Characteristics Common Examples Marketing Examples Permissible Statistics Descriptive Inferential Nominal Numbers identify and classify objects Social security numbers, numbering of football players Brand numbers, store types sex, classification Percentages, mode Chi-square, binomial test Ordinal Numbers indicate the relative positions of the objects but not the magnitude of differences between them Quality rankings, ranking of teams in tournament Preference rankings, market position, social class Percentile, median Rank-order correlation, Friedman ANOVA Interval Differences between objects can be compared; zero point is arbitrary Temperature (Fahrenheit, Centigrade) Attitudes, opinions, index numbers Range, mean, standard deviation Product-moment correlations, t-tests, ANOVA, regression, factor analysis Ratio Zero point is fixed; ratios of scale values can be compared Length, weight, time, money Age, income, costs, sales, market shares Geometric mean, harmonic mean Coefficient of variation 23
  24. 24. Paul Marx | Principles of survey research Classification of Scaling Techniques Scaling Techniques Comparative Scales Paired Comparison Rank Order Constant Sum Q-Sort & others Non- comparative Scales Continuous Rating Scales Itemized Rating Scales Likert Semantic Differential Stapel 24
  25. 25. Paul Marx | Principles of survey research Comparison of Scaling Techniques 25 Comparative Scales • involve the direct comparison of stimulus objects. • data must be interpreted in relative terms • have only ordinal and rank- order properties Non-comparative Scales • each object is scaled independently • resulting data is generally assumed to be interval or ratio scaled - nature of the research - variability in the population - statistical considerations
  26. 26. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 26
  27. 27. Paul Marx | Principles of survey research Classification of Scaling Techniques Scaling Techniques Comparative Scales Paired Comparison Rank Order Constant Sum Q-Sort & others Non- comparative Scales Continuous Rating Scales Itemized Rating Scales Likert Semantic Differential Stapel 27
  28. 28. Paul Marx | Principles of survey research Relative Advantages of Comparative Scales 28 + small differences between stimulus objects can be detected + same known reference points for all respondents + easy to understand and to use + involve fewer theoretical assumptions + tend to reduce halo or carryover effects from one judgement to another Advantages - have only ordinal and rank-order properties ⟶ limited set of statistical methods available for analysis - data must be interpreted in relative terms - Inability to generalize beyond the set of compared objects Disadvantages
  29. 29. Paul Marx | Principles of survey research Comparative Scales: Paired Comparison 29 Respondent is presented with two objects and asked to select one according to some criterion We are going to present you with ten pairs of beer brands. For each pair, please indicate which one of the two brands of beer you would prefer to purchase. Heineken Beck’s Coors Budweiser Miller Heineken Beck’s Coors Budweiser Miller #Preferred 3 2 0 4 1 Paired Comparison
  30. 30. Paul Marx | Principles of survey research Paired Comparison Scales: Examples 30
  31. 31. Paul Marx | Principles of survey research Paired Comparison: Pros-and-Cons 31 + direct comparison and overt choice + good for blind tests, physical products, and MDS + allows for calculation of percentage of respondents who prefer one stimulus to another + can assess rank-orders of stimuli (under assumption of transitivity) + possible extensions: “no difference” alternative; graded comparison Advantages - # of comparisons grows quicker than # of stimuli (for n objects n(n-1)/2 comparisons) - presentation order bias possible - preference of A over B does not imply subject’s liking of A - little similarity to real choice situation with multiple alternatives - violations of transitivity may occur Disadvantages
  32. 32. Paul Marx | Principles of survey research > > Ordinal Data: violations of transitivity in paired comparison 32
  33. 33. Paul Marx | Principles of survey research Ordinal data: violations of transitivity when aggregating preferences 33 Respondent #1 Respondent #2 Respondent #3 Votes count Result: 2 vs 1 2 vs 1 2 vs 1 Apple is both the best and the worst alternative. Aggregated preferences of the group are inconsistent! Voting
  34. 34. Paul Marx | Principles of survey research Comparative Scales: Rank Order Scaling 34 Respondents are presented with several objects simultaneously and are asked to order or rank them according to some criterion Rank the various brands of soft drinks in order of preference. Begin by picking out the one brand that you like most and assign it a number 1. Then find the second most preferred brand and assign it a number 2. Continue this procedure until you have ranked all the brands of soft drinks in order of preference. The least preferred brand should be assigned a rank of 5. No two brands should receive the same rank number. The criterion of preference is entirely up to you. There is no right or wrong answer. Just try to be consistent. Rank Order Scaling Brand Rank Order Pepsi ______________ Coke ______________ Red Bull ______________ Mountain Dew ______________ Kvas ______________
  35. 35. Paul Marx | Principles of survey research Rank Oder Scales: Example 35 ©ExavoGmbH, exavo.de
  36. 36. Paul Marx | Principles of survey research Rank Oder Scales: Examples 36 ©ExavoGmbH, exavo.de
  37. 37. Paul Marx | Principles of survey research Rank Oder Scales: Example 37 ©ExavoGmbH, exavo.de
  38. 38. Paul Marx | Principles of survey research Rank Oder Scales: Pros-and-Cons 38 + direct comparison + more realistic than paired comparison + # of comparisons is only (n-1) + easier to understand + takes less time + no intransitive responses + can be converted to paired comparison data + good for measuring preferences of brands or attributes; conjoint analysis Advantages - preference of A over B does not imply subject’s liking of A - no zero point / separation between liking and disliking - only ordinal data - violations of transitivity may occur Disadvantages
  39. 39. Paul Marx | Principles of survey research Comparative Scales: Constant Sum Scaling 39 Respondents allocate a constant sum of units (points, dollars, chips, %) among a set of stimulus objects with respect to some criterion Below are five attributes of cars. Please allocate 100 points among the attributes so that your allocation reflects the relative importance you attach to each attribute. The more points an attribute receives, the more important the attribute is. If an attribute is not at all important, assign it zero points. If an attribute is twice as important as some other attribute, it should receive twice as many points. Constant Sum Attribute Points Speed 0 Comfort 15 Gear Type 5 Fuel Type (gasoline/diesel) 35 Price 45 sum 100
  40. 40. Paul Marx | Principles of survey research Constant Sum Scaling: Example of Analysis 40 Attribute Segment 1 Segment 2 Segment 3 Speed 0 17 53 Comfort 15 23 30 Gear Type 5 21 10 Fuel Type (gasoline/diesel) 35 12 7 Price 45 27 0 sum 100 100 100 Average response of three segments
  41. 41. Paul Marx | Principles of survey research Constant Sum Scaling: Example 41 ©ExavoGmbH, exavo.de
  42. 42. Paul Marx | Principles of survey research Constant Sum Scaling: Examples 42
  43. 43. Paul Marx | Principles of survey research Constant Sum Scaling: Pros-and-Cons 43 + allows for fine discrimination among stimulus objects without requiring too much time + ratio scaled ⟶ flexible choice of data analysis methods Advantages - results are limited to the context of stimuli scaled, i.e., not generalizable to other stimuli not included in the study - relatively high cognitive burden for respondents, esp. when # of items is large - prone to calc. errors (e.g., allocation of 108 or 94 points) Disadvantages
  44. 44. Paul Marx | Principles of survey research Comparative Scales: Q-Sort Scaling 44 A rank order procedure in which objects are sorted into piles based on similarity with respect to some criterion. Usually used to discriminate among a relatively large number (60-140) of objects quickly. The number of objects in each pile is limited, usually so that all piles imitate normal distribution. To prevent epidemics, the Ministry of Health has developed the following 25 measures recommended for implementation in hospitals. Please distribute these measures for preventing the spread of infections according to their importance using the scheme below. Please allocate only one measure per box. Q-Sort not at all important extremely important
  45. 45. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 45
  46. 46. Paul Marx | Principles of survey research Classification of Scaling Techniques Scaling Techniques Comparative Scales Paired Comparison Rank Order Constant Sum Q-Sort & others Non- comparative Scales Continuous Rating Scales Itemized Rating Scales Likert Semantic Differential Stapel 46
  47. 47. Paul Marx | Principles of survey research Non-Comparative Scales: Continuous Rating Scale 47 Respondents rate objects by placing a mark at the appropriate position on a line that runs from one extreme of the criterion variable to the other. How would you rate Wal-Mart as a department store? Continuous Rating Scale Probably the worst Probably the best Version 1 х Probably the worst Probably the best Version 2 х0 10 20 30 40 50 60 70 80 90 100 Probably the worst Probably the best Version 3 х0 20 40 60 80 100 very bad very good neither good nor bad Probably the worst Probably the best Version 4 very bad very good neither good nor bad 76
  48. 48. Paul Marx | Principles of survey research Continuous Rating Scale: Perception Analyzer 48
  49. 49. Paul Marx | Principles of survey research Itemized Rating Scales: Likert Scale 49 Requires respondents to indicate a degree of agreement or disagreement with each of a series of statements about the stimulus object within typically five to seven response categories. Listed below are different opinions about 7-Eleven. Please indicate how strongly you agree or disagree with each by using the following scale: Likert Scale Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 7-Eleven sells high-quality merchandise [1] [x] [3] [4] [5] 7-Eleven has poor in-store service [1] [x] [3] [4] [5] I like to shop in 7-Eleven [1] [2] [x] [4] [5] 7-Eleven does not offer a good mix of different brands within a product category [1] [2] [3] [x] [5] The credit policies at 7-Eleven are terrible [1] [2] [3] [x] [5] I do not like advertising done by 7-Eleven [1] [2] [3] [x] [5] 7-Eleven charges fair prices [1] [x] [3] [4] [5] NOTICE the reversed scoring of items 2,4,5, and 6. Reverse the scale for these items prior analyzing to be consistent with the whole set of items, i.e. a higher score should denote a more favorable attitude.
  50. 50. Paul Marx | Principles of survey research Likert Scale: Examples 50
  51. 51. Paul Marx | Principles of survey research Some Commonly Used Scales in Marketing 51 Construct Scale Descriptors Attitude Very bad Bad Neither Bad Nor Good Good Very Good Importance Not at All Important Not Important Neutral Important Very Important Satisfaction Very Dissatisfied (Somewhat) Dissatisfied Neither Dissatisfied Nor Satisfied / Neutral (Somewhat) Satisfied Very Satisfied Purchase Intention Definitely Will Not Buy Probably will Not Buy Might or Might Not Buy Probably Will Buy Definitely Will Buy Purchase Frequency Never Rarely Sometimes Often Very Often Agreement Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree
  52. 52. Paul Marx | Principles of survey research Itemized Rating Scales: Semantic Differential 52 A rating scale with end point associated with bipolar labels that have semantic meaning. Respondents are to indicate how accurately or inaccurately each term describes the object. This part of the study measures what certain department stores mean to you by having you judge them on a series of descriptive scales bounded at each end by one of two bipolar adjectives. Please mark (X) the blank that best indicates how accurately one or the other adjective describes what the store means to you. Please be sure to mark every scale; do not omit any scale.Semantic Differential Powerful [ ] [ ] [ ] [ ] [X] [ ] [ ] Weak Unreliable [ ] [ ] [ ] [ ] [ ] [X] [ ] Reliable Modern [ ] [ ] [ ] [ ] [ ] [ ] [X] Old fashioned Cold [ ] [ ] [ ] [ ] [ ] [X] [ ] Warm Careful [ ] [X] [ ] [ ] [ ] [ ] [ ] Careless NOTE: The negative adjective sometimes appears at the left side of the scale and sometimes at the right. This controls the tendency of some respondents, particularly those with very positive or very negative attitudes, to mark the right- or left-hand sides without reading the labels. 7-Eleven is:
  53. 53. Paul Marx | Principles of survey research Semantic Differential Scale: Example 53 Rugged [ ] [ ] [ ] [ ] [ ] [ ] [ ] Delicate Excitable [ ] [ ] [ ] [ ] [ ] [ ] [ ] Calm Uncomfortable [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comfortable Dominating [ ] [ ] [ ] [ ] [ ] [ ] [ ] Submissive Thrifty [ ] [ ] [ ] [ ] [ ] [ ] [ ] Indulgent Pleasant [ ] [ ] [ ] [ ] [ ] [ ] [ ] Unpleasant Contemporary [ ] [ ] [ ] [ ] [ ] [ ] [ ] Non-contemporary Organized [ ] [ ] [ ] [ ] [ ] [ ] [ ] Unorganized Rational [ ] [ ] [ ] [ ] [ ] [ ] [ ] Emotional Youthful [ ] [ ] [ ] [ ] [ ] [ ] [ ] Mature Formal [ ] [ ] [ ] [ ] [ ] [ ] [ ] Informal Orthodox [ ] [ ] [ ] [ ] [ ] [ ] [ ] Liberal Complex [ ] [ ] [ ] [ ] [ ] [ ] [ ] Simple Colorless [ ] [ ] [ ] [ ] [ ] [ ] [ ] Colorful Modest [ ] [ ] [ ] [ ] [ ] [ ] [ ] Vain Measuring Self-Concepts, Person Concepts, and Product Concepts Rating profiles of different objects / respondents / segments. Each point corresponds to a mean or median of the respective scale.
  54. 54. Paul Marx | Principles of survey research Semantic Differential Scale: Example 54 Source: http://www.provisor.com.ua/archive/2000/N16/gromovik.php Cheap [ ] [ ] [ ] [ ] [ ] [ ] [ ] Expensive Has natural ingredients [ ] [ ] [ ] [ ] [ ] [ ] [ ] Has no natural ingredients Attractive [ ] [ ] [ ] [ ] [ ] [ ] [ ] Unattractive Easily available [ ] [ ] [ ] [ ] [ ] [ ] [ ] Hard to get Smells good [ ] [ ] [ ] [ ] [ ] [ ] [ ] Smells bad Has conditioner [ ] [ ] [ ] [ ] [ ] [ ] [ ] Has no conditioner Well-known brand [ ] [ ] [ ] [ ] [ ] [ ] [ ] Unknown brand Suitable for frequent usage [ ] [ ] [ ] [ ] [ ] [ ] [ ] Unsuitable for frequent usage Miraculous effect of cleanliness and shine [ ] [ ] [ ] [ ] [ ] [ ] [ ] Lack of cleanliness effect Easy-to-use [ ] [ ] [ ] [ ] [ ] [ ] [ ] Inconvenient to use Ideal shampoo Elseve Herbal Magic Semantic profiles of shampoo brands “Herbal Magic” and “Elseve” in comparison with an ideal shampoo from consumers’ point of view
  55. 55. Paul Marx | Principles of survey research Semantic Differential Scale: Example 55
  56. 56. Paul Marx | Principles of survey research Itemized Rating Scales: Stapel Scale 56 An unipolar rating scale with 10 categories numbered from -5 to +5 without neutral point (zero). Used as an alternative to semantic differential, especially when a meaningful pair of opposed adjectives is difficult to construct. Please evaluate how accurately each word or phrase describes each of department stores. Select a plus number for phrases you think describe the store accurately. The more accurately you think the phrase describes the store, the larger the plus number you should choose. You should select a minus number for phrases you think do not describe it accurately. The less accurately you think the phrase describes the store, the larger the minus number you should choose. You can select any number, from +5 for phrases you think are very accurate, to -5 for phrases you think are very inaccurate. Stapel Scale 7-Eleven: +5 +4 +3 +2 +1 -1 -2 -3 -4 -5 High Quality +5 +4 +3 +2 +1 -1 -2 -3 -4 -5 Poor service х х
  57. 57. Paul Marx | Principles of survey research Basic Non-Comparative Scales Scale Basic Characteristics Examples Advantages Disadvantages Continuous Rating Scale Place a mark on a continuous line Reaction to TV commercials Easy to construct Scoring can be cumbersome, unless computerized Itemized Scales Likert 
Scale Degrees of agreements on a 1 (strongly disagree) to 5 (strongly agree) scale Measurement of attitudes Easy to construct, administer and understand More time-consuming Semantic Differential Seven-point scale with bipolar labels Brand, product, and company images Versatile Controversy as to whether the data are interval Stapel Scale Unipolar ten-point scale, -5 to +5, without a neutral point (zero) Measurement of attitudes and images Easy to construct, administer over telephone Confusing an difficult to apply 57
  58. 58. Paul Marx | Principles of survey research Non-comparative Itemized Rating Scale Decisions 58 Number of categories Although there is no single, optimal number, traditional guidelines suggest that there should be between five and nine categories. Balanced vs. unbalanced In general, the scale should be balanced to obtain objective data. Odd/even no. of categories If a neutral or indifferent scale response is possible for at least some respondents, an odd number of categories should be used. Forced vs. non-forced In situations where the respondents are expected to have no opinion, the accuracy of the data may be improved by a non- forced scale. Verbal description An argument can be made for labeling all or many scale categories. The category descriptions should be located as close to the response categories as possible.
  59. 59. Paul Marx | Principles of survey research Number of categories Although there is no single, optimal number, traditional guidelines suggest that there should be between five and nine categories. Number of Scale Categories 59 + The greater the number of scale categories, the finer the discrimination among stimulus objects that is possible - Most respondents cannot handle more than a few categories Involvement and knowledge • more categories when respondents are interested in the scaling task or are knowledgeable about the objects Nature of the objects • do objects lend themselves to fine discrimination? Mode of data collection • less categories in telephone interviews Data analysis • less categories for aggregation, broad generalizations or group comp. • more categories for sophisticated statistical analysis, esp. correlation based ones
  60. 60. Paul Marx | Principles of survey research Balanced vs. unbalanced In general, the scale should be balanced to obtain objective data. Balanced vs. Unbalanced Scales 60 Extremely good Very good Neither good nor bad Very bad Extremely bad Balanced Scale Extremely good Very good Good Somewhat good Bad Very bad Unbalanced Scale
  61. 61. Paul Marx | Principles of survey research Odd/even no. of categories If a neutral or indifferent scale response is possible for at least some respondents, an odd number of categories should be used. Odd or Even Number of Categories 61 - The middle option of an attitudinal scale attracts a substantial # of respondents who might be unsure about their opinion or reluctant to disclose it - This can distort measures of central tendency and variance - Do we want/need “contrast” in controversial attitudes?
  62. 62. Paul Marx | Principles of survey research Forced vs. non-forced In situations where the respondents are expected to have no opinion, the accuracy of the data may be improved by a non- forced scale. Forced vs. Non-Forced 62 - Questions that exclude the "don't know" option tend to produce a greater volume of accurate data - Are respondents unwilling to answer vs. don’t have an opinion? - Use "don't know" or better “not applicable” option for factual questions, but not for attitude questions - Use branching to ensue concept familiarity on the respondent’s side
  63. 63. Paul Marx | Principles of survey research Verbal description An argument can be made for labeling all or many scale categories. The category descriptions should be located as close to the response categories as possible. Verbal Description 63 - Providing a verbal description for each category may not improve the accuracy or reliability of the data vs. scale ambiguity - Peaked vs. flat response distributions completely disagree completely agree disagree agree
  64. 64. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 64
  65. 65. Paul Marx | Principles of survey research Latent Constructs 65 Please indicate how satisfied you were with your purchase of _____ by checking the space that best gives your answer. satisfied [ ] [ ] [ ] [ ] [ ] [ ] [ ] dissatisfied pleased [ ] [ ] [ ] [ ] [ ] [ ] [ ] displeased favorable [ ] [ ] [ ] [ ] [ ] [ ] [ ] unfavorable pleasant [ ] [ ] [ ] [ ] [ ] [ ] [ ] unpleasant I like it very much [ ] [ ] [ ] [ ] [ ] [ ] [ ] I didn't like it at all contented [ ] [ ] [ ] [ ] [ ] [ ] [ ] frustrated delighted [ ] [ ] [ ] [ ] [ ] [ ] [ ] terrible α=0,84 A Latent Construct is a variable that cannot be observed or measured directly but can be inferred from other observable measurable variables. Thus, the researcher must capture the variable through questions representing the presence/level of the variable in question.
  66. 66. Paul Marx | Principles of survey research Latent Constructs & Multi-Item Scales Construct Dimensions Factors Items Scale customer satisfaction satisfaction with product satisfaction with service friendliness expertise liability the salesperson was appealing the salesperson smiled to me the salesperson was courteous strongly agree largely agree largely disagree strongly disagree
  67. 67. Paul Marx | Principles of survey research Advantages + allow to assess abstract concepts + make it easier to understand the data and phenomenon + reduce dimensionality of data through aggregating a large number of observable variables in a model to represent an underlying concept + link observable (“sub-symbolic”) data of the real world to symbolic data in the modeled world Latent Constructs & Multi-Item Scales 67
  68. 68. Paul Marx | Principles of survey research Multi-Item Scales: Make or Steal Generate an initial pool of items: theory, secondary data, and qualitative research Select a reduced set of items based on qualitative judgement Collect data from a large pretest sample Perform statistical analysis Develop a purified scale Collect more date form a different sample Evaluate scale reliability, validity, and generalizability Prepare the final scale Develop a theory Brunner, Gordon C. II (2012), “Marketing Scales Handbook: A Compilation of Multi-Item Measures for Consumer Behavior & Advertising Research”, Vol. 6, available as PDF at www.marketingscales.com/research Journal of the Academy of Marketing Science (JAMS) Journal of Advertising (JA) Journal of Consumer Research (JCR) Journal of Marketing (JM) Journal of Marketing Research (JMR) Journal of Retailing (JR)
  69. 69. Paul Marx | Principles of survey research Secure Customer Index™ Assessing Consumer Loyalty and Retention 69 Secure Customer Very satisfied Definitely would recommend Definitely will use again D. Randall Brandt (1996), “Secure Customer Index”, Maritz Research Overall Satisfaction 4 = very satisfied 3 = somewhat satisfied 2 = somewhat dissatisfied 1 = very dissatisfied Willingness to Recommend 5 = definitely would recommend 4 = probably would recommend 3 = might or might not recommend 2= probably would not recommend 1= definitely would not recommend Likelihood to Use Again 5 = definitely will use again 4 = probably will use again 3= might or might not use again 2= probably will not use again 1 = definitely will not use again Secure Customers % very satisfied/definitely would repeat/definitely would recommend Favorable Customers % giving at least "second best" response on all three measures of satisfaction and loyalty Vulnerable Customers % somewhat satisfied/might or might not repeat/might or might not recommend At Risk Customers % somewhat satisfied or dissatisfied/probably or definitely would not repeat/probably or definitely would not recommend
  70. 70. Paul Marx | Principles of survey research Extended Secure Customer Index™ Burke Inc. 70 Overall Satisfaction What is your overall level of satisfaction with (BRAND/CO)? Willingness to Recommend If you were asked to recommend a (INDUSTRY) how likely would you be to recommend (BRAND/CO.)? Likelihood to Repurchase How likely are you to continue using (BRAND/CO.)? Earned Loyalty (BRAND/CO.) has earned my loyalty Preferred Company I prefer (BRAND/CO.) to all other providers Burke Inc. http://www.burke.com/library/whitepapers/sci_white_paper_low_res_pages.pdf Loyalty Index Share of Wallet (0% - 100%) Period 1 Period 2
  71. 71. Paul Marx | Principles of survey research 2.Survey: Measurement and Scaling 2.1 Introduction 2.2 Comparative Scales 2.3 Non-Comparative Scales 2.4 Multi-item Scales 2.5 Reliability and Validity 71
  72. 72. Paul Marx | Principles of survey research Multi-Item Scales: Measurement Accuracy 72 The True Score Model ХO = ХT + ХS + ХR where ХO = the observed score of measurement ХT = the true score of characteristic ХS = systematic error ХR = random error
  73. 73. Paul Marx | Principles of survey research Reliability & Validity 73 Reliability • extent to which a scale produces consistent results in repeated measurements • absence of random error (ХR ⟶0 |⇒ ХO ⟶ ХT + ХS) • reliability of a multi-item scale is denoted as Cronbach’s alpha (0 ≥ α ≥ 1) • values of α ≥ 0,7 are considered satisfactory ХO = ХT + ХS + ХR Validity • extent to which differences in observed scale scores reflect true differences among objects on the characteristic being measured • no measurement error (ХS ⟶ 0, ХR ⟶ 0 |⇒ ХO ⟶ХT) Reliable Not Valid Low Validity Low Reliability Not Reliable Not Valid Both Reliable and Valid * α can take on also negative values, however, they cannot be interpreted
  74. 74. Paul Marx | Principles of survey research Reliable Not Valid Low Validity Low Reliability Not Reliable Not Valid Both Reliable and Valid Relationship between Reliability & Validity 74 ХO = ХT + ХS + ХR • validity implies reliability (ХO = ХT |⇒ ХS = 0, ХR = 0) • unreliability implies invalidity (ХR ≠ 0 |⇒ ХO = ХT + ХR ≠ ХT) • reliability does not imply validity (ХR = 0, ХS ≠ 0 |⇒ ХO = ХT + ХS ≠ ХT) • reliability is a necessary, but not sufficient, condition of validity
  75. 75. Paul Marx | Principles of survey research 75 “The purpose of a scale is to allow us to represent respondents with the highest accuracy and reliability. We can’t have one without the other and still believe in our data.” Bart Gamble vice president client services, Burke, Inc. 2000-2003
  76. 76. Paul Marx | Principles of survey research Net Promoter Score® competitive growth rates? 76 0 1 2 3 4 5 6 7 8 9 10 Reichheld, Fred (2003) "One Number You Need to Grow", Harvard Business Review Detractors Passives Promoters Net Promoter Score % Promoters % Detractors= – How likely are you to recommend company/brand/product X to a friend/colleague/relative? Is the scale reliable? Is the scale valid? NPS (-100% – +100%) 5-10% average companies 45% high potentials with open growth opportunity 50-80% market leaders with high growth potential
  77. 77. Paul Marx | Principles of survey research Net Promoter Score®: Warning 77 “Though the “would recommend” question is far and away the best single-question predictor of customer behavior across a range of industries, it’s not the best for every industry…So, companies need to do their homework. They need to validate the empirical link between survey answers and subsequent customer behavior for their own business.” Fred Reichheld, 2011 Reichheld, Fred, with Rob Markey (2011). The Ultimate Question 2.0. Boston: Harvard Business Review Press; pp.50-51. ?
  78. 78. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 78
  79. 79. Paul Marx | Principles of survey research Questionnaire 79 A Questionnaire – is a formalized set of questions for obtaining information from respondents. Objectives of a Questionnaire: • translate the information need into a set of specific questions that the respondents can and will answer • uplift, motivate, and encourage respondents to become involved in the interview, to cooperate, and to complete the interview • minimize response error Questionnaire
  80. 80. Paul Marx | Principles of survey research Questioning Tactics 80 • Choose an answer form a list of answer choices • +: easy to analyze, do not task respondents’ memory and make less stress • –: automatic and snap answers • Response options are not set • +: unlimited range of possible responses, “tests” respondent’s memory • –: complexity of coding and analysis, respondents may refuse to answer Closed-ended Open ended • Do you drink alcohol every day? • What drinks do you prefer for dinner? Direct Indirect
  81. 81. Paul Marx | Principles of survey research Bias in Formulation 81 Q: Do you approve smoking whilst praying? A: No Q: Do you approve praying whilst smoking? A: Yes 0 15 30 45 60 Yes No Uncertain Do you actually believe in the big love? Do you believe in the big love? Noelle-Neumann and Petersen (1998), p. 192 n = 2100, p <.05
  82. 82. Paul Marx | Principles of survey research Issues to Consider in Questionnaire Design 82 • Is the question necessary? • Are several questions needed instead of one? • Is the respondent informed? • Can the respondent remember? • Effort required of the respondents • Sensitivity of question • Legitimate purpose • Cultural issues • Ease of completion • Comprehensiveness • Bias in formulation
  83. 83. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 83
  84. 84. Paul Marx | Principles of survey research Asking Questions 84 “It is not every question that deserves an answer” Publius Syrus roman, 1st century B.C. • Avoid ambiguity, confusion, and vagueness • Avoid jargon, slang, abbreviations • Avoid double-barreled questions • Avoid leading • Avoid implicit assumptions • Avoid implicit alternatives • Avoid treating respondent’s belief about a hypothesis as a test of the hypothesis • Avoid generalizations and estimates
  85. 85. Paul Marx | Principles of survey research Avoid Ambiguity, Confusion and Vagueness 85 Define the issue in terms of who, what, when, where, why, and way (the six Ws). Who, what, when, and where are particularly important. • Example: Which brand of shampoo do you use? • Ask instead: Which brand or brands of shampoo have you personally used at home during the last month? In case of more than one brand, please list all the brands that apply.
  86. 86. Paul Marx | Principles of survey research Avoid Ambiguity, Confusion and Vagueness 86 The W’s Defining the Question Who The Respondent It is not clear whether this question relates to the individual respondent or, e.g., the respondent’s total household What The Brand of Shampoo It is unclear how the respondent is to answer this question if more than one brand is used When Unclear The time frame is not specified in this question. The respondent could interpret it as meaning the shampoo used this morning, this, week, or over the past year. Where Unclear At home, at gym, on the road? Which brand of shampoo do you use?
  87. 87. Paul Marx | Principles of survey research Avoid Ambiguity, Confusion and Vagueness 87 • Example: What brand of computer do you own? ☐ Windows ☐ Mac OS • Ask instead: Do you own a Windows PC? (☐ Yes ☐ No) Do you own an Apple computer? (☐ Yes ☐ No) • Even better: What brand of computer do you own? ☐ Do not own a computer ☐ Windows ☐ Mac OS ☐ Other • Example: Are you satisfied with your current auto insurance? ☐ Yes ☐ No • Ask instead: Are you satisfied with your current auto insurance? ☐ Yes ☐ No ☐ Don’t have auto insurance • Even better (branch questions): 1. Do you currently have a life insurance policy? (☐ Yes ☐ No). If no, go to question 3. 2. Are you satisfied with your current auto insurance? (☐ Yes ☐ No)
  88. 88. Paul Marx | Principles of survey research Avoid Ambiguity, Confusion and Vagueness 88 Example: In a typical month, how often do you shop in department stores? ☐ Never ☐ Occasionally ☐ Sometimes ☐ Often ☐ Regularly • Ask instead: In a typical month, how often do you shop in department stores? ☐ Less than once ☐ 1 or 2 times ☐ 3 or 4 times ☐ More than 4 times Whenever using words “will”, “could”, “might”, or “may” in a question, you might suspect that the question asks a time-related question.
  89. 89. Paul Marx | Principles of survey research Avoid Jargon, Slang, Abbreviations 89 Use ordinary words • Example: Do you think the distribution of soft drinks is adequate? • Ask instead: Do you think soft drinks are readily available when you want to buy them? • Example: What was your AGI last year? $ _______
  90. 90. Paul Marx | Principles of survey research Avoid Double-Barreled Questions 90 Are several questions needed instead of one? • Example: Do you think Coca-Cola is a tasty and refreshing soft drink? • Ask instead: 1. Do you think Coca-Cola is a tasty soft drink? 2. Do you think Coca-Cola is a refreshing soft drink?
  91. 91. Paul Marx | Principles of survey research Avoid Leading 91 If you want a certain answer - why ask? • Example: Do you help the environment by using canvas shopping bags? • Ask instead: Do you use canvas shopping bags?
  92. 92. Paul Marx | Principles of survey research Avoid Implicit Assumptions 92 The answer should not depend on upon implicit assumptions about what will happen as a consequence. • Example: Are you in favor of a balanced budget? • Ask instead: Are you in favor of a balanced budget if it would result in an increase in the personal income tax?
  93. 93. Paul Marx | Principles of survey research http://www.kostenlose3dmodelle.com/ mensch-argere-dich-nicht-lightwavedice -studio-3ds-obj-lwo/ Avoid implicit alternatives 93 An alternative that is not explicitly expressed in the options is an implicit alternative. • Example: Do you like to fly when traveling short distances? • Ask instead: Do you like to fly when traveling short distances, or would you rather drive?
  94. 94. Paul Marx | Principles of survey research Avoid Treating Beliefs as Real Facts 94 Beliefs are only a biased representation of reality • Example: Do you think more educated people wear fur clothing? • Ask instead: 1. What is your education level? 2. Do you wear fur clothing?
  95. 95. Paul Marx | Principles of survey research Avoid Generalizations and Estimates 95 Don’t task respondents’ memory and math skills • Example: What is the annual per capita expenditure on groceries in your household? • Ask instead: 1. What is the monthly (or weekly) expenditure on groceries in your household? 2. How many members are there in your household?
  96. 96. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 96
  97. 97. Paul Marx | Principles of survey research Overcoming Inability to Answer 97 Is the Respondent Informed? Can the Respondent Remember? Can the Respondent Articulate?
  98. 98. Paul Marx | Principles of survey research Overcoming Inability to Answer 98 Is the Respondent Informed? Respondents will often answer questions even though they are not informed • Example: Please indicate how strongly you agree or disagree with the following statement: “The National Bureau of Consumer Complaints provides an effective means for consumers who have purchased a defective product to obtain relief” 51.9% of the lawyers and 75% of the public expressed their opinion, although there is no such entity as the NBCC • Use Filter Questions: e.g. ask about familiarity and/or frequency of patronage in a study of 10 department stores • Use “don’t know” Option
  99. 99. Paul Marx | Principles of survey research Can the Respondent Remember? Overcoming Inability to Answer 99 The inability to remember leads to errors of omission, telescoping, and creation • Example: How many liters of soft drinks did you consume during the last four weeks? • Ask instead: How often do you consume soft drinks in a typical week? ☐ Less than once a week ☐ 1 to 3 times per week ☐ 4 or 6 times per week ☐ 7 or more times per week • Use aided recall approach (where appropriate) “What brands of soft drinks do you remember being advertised last night on TV?” vs “Which of these brands were advertised last night on TV?”
  100. 100. Paul Marx | Principles of survey research Can the Respondent Articulate? Overcoming Inability to Answer 100 If unable to articulate their responses, respondents are likely to ignore the question and quit the survey • Example: If asked to describe the atmosphere of the department store they would prefer to patronage, most respondents may be unable to phrase their answers. • Provide aids, e.g., pictures, maps, descriptions If the respondents are given alternative descriptions of store atmosphere, they will be able to indicate the one they like the best.
  101. 101. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 101
  102. 102. Paul Marx | Principles of survey research Overcoming Unwillingness to Answer 102 Most respondents are unwilling to • devote a lot of effort to provide information • respond to questions that they consider to be inappropriate for the given context • divulge information they do not see as serving a legitimate purpose • disclose sensitive information
  103. 103. Paul Marx | Principles of survey research Overcoming Unwillingness to Answer 103 Minimize the effort required of respondents • Example: Please list all the departments from which you purchased merchandise on your most recent shopping to a department store. • Ask instead: In the list that follows, please check all the departments from which you purchased merchandise on your most recent shopping to a department store. ☐ Women’s dresses ☐ Men’s apparel ☐ Children’s apparel ☐ Cosmetics ……. ☐ Jewelry ☐ Other (please specify) _________________
  104. 104. Paul Marx | Principles of survey research Overcoming Unwillingness to Answer 104104 Some questions may seem appropriate in certain contexts but not in others • Example: Questions about personal hygiene habits may be appropriate when asked in a survey sponsored by the Medical Association, but not in one sponsored by a fast- food restaurant. • Provide context by making a statement: “As a fast-food restaurant, we are very concerned about providing a clean and hygienic environment for our customers. Therefore, we would like to ask you some questions related to personal hygiene.”
  105. 105. Paul Marx | Principles of survey research Overcoming Unwillingness to Answer 105105105 Explain why the data is needed • Example: Why should a firm marketing cereals want to know the respondents’ age, income, and occupation? • Legitimate the information request: “To determine how the consumption of cereals vary among people of different ages, incomes, and occupation, we need information on ...”
  106. 106. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 106
  107. 107. Paul Marx | Principles of survey research • Place sensitive topics at the end of the questionnaire • Preface questions with a statement that the behavior is of interest in common • Ask the question using third-person technique: phrase the question as if it referred to other people • Hide the question in a group of other questions • Provide response categories rather than asking for specific figures Increasing Willingness of Respondents 107 Sensitive Topics: - money - family life - political and religious beliefs - involvement in accidents or crimes - …
  108. 108. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 108
  109. 109. Paul Marx | Principles of survey research Determining the Order of Questions 109 • Opening Questions The opening questions should be interesting, simple, and non-threatening. • Type of Information As a general guideline, basic information should be obtained first, followed by classification, and, finally, identification information. • Difficult Questions Difficult questions or questions which are sensitive, embarrassing, complex, or dull, should be placed late in the sequence.
  110. 110. Paul Marx | Principles of survey research Determining the Order of Questions 110 • Effect on Subsequent Questions (funneling) General questions should precede the specific questions 1. What considerations are important to you in selecting a department store? 2. In selecting a department store, how important is convenience of location? • Logical Order / Branching Questions The question being branched should be placed as close as possible to the question causing the branching. The branching questions should be ordered so that the respondents cannot anticipate what additional information will be required.
  111. 111. Paul Marx | Principles of survey research Example: Flowchart of a Questionnaire 111 Introduction Ownership of Store, Bank, and/or other Charge Cards Purchased products in a specific department store during the last two months How payment was made? Ever purchased products in a department store? Store Charge Card Bank Charge Card Other Charge Card Intention to use Store, Bank, or Other Charge Cards yes no yes no Credit Cash Other
  112. 112. Paul Marx | Principles of survey research 3.Questionnaire 3.1 Asking Questions 3.2 Overcoming Inability to Answer 3.3 Overcoming Unwillingness to Answer 3.4 Increasing Willingness of Respondents 3.5 Determining the Order of Questions 3.6 What’s Next? 112
  113. 113. Paul Marx | Principles of survey research What’s Next? 113113 Introduction • Catch the respondents’ interest • Explain the reasons & objectives • Ask for their help • Tell that their support is valuable • Tell how much time it will last • Emphasize the anonymity • Incentivize (non-monetary incentives)
  114. 114. Paul Marx | Principles of survey research What’s Next? 114114 Pretest! Pretest! Pretest!!! • question content • wording • sequence • form and layout • question difficulty • instructions… • analysis procedures
  115. 115. Paul Marx | Principles of survey research Recap 115 1. Develop a flow chart of the information required based on the marketing research problem • Once the entire sequence is laid out, the interrelationships should become clear • Match up the actual data you would expect to collect from the questionnaire against the information needs listed in the flow chart • Be specific in the objective for each area of information and data. You should be able to write an objective for each area so specifically that it guides your construction of the questions. 2. At this stage, put on your “critic’s” hat on and go back over the flowchart and ask • Do I need to know it and know exactly what I am going to do with it? or • It would be nice to know it but I do not have to have it
  116. 116. Paul Marx | Principles of survey research 4.Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 116
  117. 117. Paul Marx | Principles of survey research 117 The world’s most famous newspaper error President Harry Truman against Thomas Dewey Chicago Tribute prepared an incorrect headline without first getting accurate information Reason? • bias • inaccurate opinion polls
  118. 118. Paul Marx | Principles of survey research Sampling 118 Most research cannot test everyone. Instead a sample of the whole population is selected and tested. If this is done well, the results can be applied to the whole population. This selection and testing of a sample is called sampling. If a sample is poorly chosen, all the data may be useless. Population the group of people we wish to understand. Populations are often segmented by demographic or psychographic features (age, gender, interests, lifestyles, etc.) Sample a subset of population that represents the whole group
  119. 119. Paul Marx | Principles of survey research Sampling 119 Population the group of people we wish to understand. Populations are often segmented by demographic or psychographic features (age, gender, interests, lifestyles, etc.) Sample a subset of population that represents the whole groupRespondents people who answer Most research cannot test everyone. Instead a sample of the whole population is selected and tested. If this is done well, the results can be applied to the whole population. This selection and testing of a sample is called sampling. If a sample is poorly chosen, all the data may be useless.
  120. 120. Paul Marx | Principles of survey research Sampling: Two General Methods 120 Image By Sergio Valle Duarte (Own work) [CC BY 3.0], via Wikimedia Commons
  121. 121. Paul Marx | Principles of survey research 121 Sampling Techniques Convenience Sampling Judgmental Sampling Quota Sampling Snowball Sampling Non-probability Probability Simple Random Sampling Systematic Sampling Stratified Sampling Cluster Sampling Other Sampling Techniques Proportionate Disproportionate
  122. 122. Paul Marx | Principles of survey research 4.Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 122
  123. 123. Paul Marx | Principles of survey research 123 Sampling Techniques Convenience Sampling Judgmental Sampling Quota Sampling Snowball Sampling Non-probability Probability Simple Random Sampling Systematic Sampling Stratified Sampling Cluster Sampling Other Sampling Techniques Proportionate Disproportionate
  124. 124. Paul Marx | Principles of survey research Convenience Sampling 124 Convenience sampling attempts to obtain a sample of convenient respondents. Often, respondents are selected because they happen to be in the right place at right time. • students or members of social organizations • mall intercept interviews without qualifying the respondents • “people on the street” interviews • tear-out questionnaires in magazines
  125. 125. Paul Marx | Principles of survey research Judgmental Sampling 125 Judgmental sampling a form of convenience sampling in which the population elements are selected based on the judgement of the researcher • test markets • purchase engineers selected in industrial marketing research • mothers as diaper “users”
  126. 126. Paul Marx | Principles of survey research Quota Sampling 126 Quota sampling techniques develop control categories, or quotas, of population elements (e.g., sex, age, race, income, company size, turnover, etc.) so that the proportion of the elements possessing these characteristics in the sample reflects their distribution in the population. The elements themselves are selected based on convenience or judgment. The only requirement, however, is that the elements selected fit the control characteristics (quota). Control Characteristic Population Composition Sample Composition Percentage Percentage Number Sex Male
 Female 
 48 52 ------- 100 48
 52
 ------- 100 
 480
 520
 ------- 1000 Age
 18-30 31-45 45-60
 Over 60 27 39 16 18 ------- 100 27 39 16 18 ------- 100 270 390 160 180 ------- 1000
  127. 127. Paul Marx | Principles of survey research Snowball Sampling 127127 An initial group of respondents is selected (usually) at random. • After being interviewed, these respondents are asked to identify others who belong to the target population of interest. • Subsequent respondents are selected based on the referrals. Good for locating the desired characteristic in the population: • reaching hard-to-reach respondents (e.g., government services, “food stamps”, drug users) • estimating characteristics that are rare in the population • identifying buyer-seller pairs in industrial research
  128. 128. Paul Marx | Principles of survey research 4.Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 128
  129. 129. Paul Marx | Principles of survey research 129 Sampling Techniques Convenience Sampling Judgmental Sampling Quota Sampling Snowball Sampling Non-probability Probability Simple Random Sampling Systematic Sampling Stratified Sampling Cluster Sampling Other Sampling Techniques Proportionate Disproportionate
  130. 130. Paul Marx | Principles of survey research Simple Random Sampling & Systematic Sampling 130 Systematic Sampling • The sample is chosen by selecting a random starting point and then picking every 𝑖-th element in succession from the sampling frame • The sampling interval, 𝑖, is determined by dividing the population size 𝑁 by the sample size 𝑛, i.e., 𝑖 = 𝑁/𝑛 Simple Random Sampling • Each element in the population has a known and equal probability of selection • Each possible sample of a given size (𝑛) has a known probability of being the sample actually selected • This implies that every element is selected independently of every other element. start here select randomly i i i take every i-th element
  131. 131. Paul Marx | Principles of survey research Stratified Sampling 131131 Stratified sampling is obtained by separating the population into non-overlapping groups called strata and then obtaining a proportional simple random sample from each group. The individuals within each group should be similar in some way. Good for: • highlighting a specific subgroup within the population • observing existing relationships between two or more subgroups • representative sampling of even the smallest and most inaccessible subgroups in the population • a higher statistical precision Stratum A B C Population Size 100 200 300 Sampling Fraction 1/2 1/2 1/2 Final Sample Size 50 100 150 Stratum A B C Population Size 100 200 300 Sampling Fraction 1/5 1/2 1/3 Final Sample Size 20 100 100 Proportionate Disproportionate
  132. 132. Paul Marx | Principles of survey research Cluster Sampling 132132 Cluster sampling the target population is first divided into mutually exclusive and collectively exhaustive subpopulations, or clusters. Than a random sample of clusters is selected, based on SRS. Good for: • covering large geographic areas • reducing survey costs • when constructing a complete list of population elements is difficult • when the population concentrated in natural clusters (e.g., blocks, cities, schools, hospitals, boxes, etc.) For each cluster, either all the elements are included in the sample (one-stage) or a sample of elements is drawn probabilistically (two-sage).
  133. 133. Paul Marx | Principles of survey research 4.Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 133
  134. 134. Paul Marx | Principles of survey research Strengths and Weaknesses of Basic Sampling Techniques 134 Technique Strengths Weaknesses Non-probability Sampling Convenience sampling Least expensive, least time consuming, most convenient Selection bias, sample not representative, not recommended for descriptive or causal research Judgmental sampling Low cost, convenient, not time consuming Does not allow generalization, subjective Quota sampling Sample can be controlled for certain characteristics Selection bias, no assurance of representativeness Snowball sampling Can estimate rare characteristics Time consuming in the field research Probability Sampling Simple random sampling (SRS) Easily understood, results projectable Difficult to construct sampling frame, expensive, lower precision, no assurance of representativeness Systematic sampling Can increase representativeness, easier to implement than SRS Can decrease representativeness Stratified sampling Includes all important subpopulations, precision Difficult to select relevant stratification variables, not feasible to stratify on many variables, expensive Cluster sampling Easy to implement, cost effective Imprecise, difficult to compute and interpret results
  135. 135. Paul Marx | Principles of survey research 4.Sampling 4.1 Non-probability Sampling 4.2 Probability Sampling 4.3 Choosing Non-probability vs. Probability Sampling 4.4 Sample Size 135
  136. 136. Paul Marx | Principles of survey research Determining the Sample Size 136 The sample size does not depend on the size of the population being studied, but rather it depends on qualitative factors of the research. • desired precision of estimates • knowledge of population parameters • number of variables • nature of the analysis • importance of the decision • incidence and completion rates • resource constraints
  137. 137. Paul Marx | Principles of survey research Sample Sizes Used in Marketing Research Studies 137 Type of Study Minimum Size Typical Size Problem identification research (e.g., market potential) 500 1,000 - 2,000 Problem solving research (e.g., pricing) 200 300 - 500 Product tests 200 300 - 500 Test-market studies 200 300 - 500 TV/Radio/Print advertising (per commercial ad tested) 150 200 - 300 Test-market audits 10 stores 10 - 20 stores Focus groups 6 groups 10 - 15 groups
  138. 138. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 138
  139. 139. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 139
  140. 140. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 140
  141. 141. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 141 𝑥 = 𝑥( ± 𝐸 𝑥 = real population parameter 𝑥( = sample statistic 𝐸 = margin of error 𝐸 = 𝑧 𝜎 𝑛
  142. 142. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 142 𝑥 = 𝑥( ± 𝐸 𝑥 = real population parameter 𝑥( = sample statistic 𝐸 = margin of error 𝐸 = 𝑧 𝜎 𝑛 unlikely to be known
  143. 143. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 143 𝑥 = 𝑥( ± 𝐸 𝑥 = real population parameter 𝑥( = sample statistic 𝐸 = margin of error 𝐸 = 𝑧 𝜎 𝑛 unlikely to be known has a maximum at π = .5
  144. 144. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 144 𝑥 = 𝑥( ± 𝐸 𝑥 = real population parameter 𝑥( = sample statistic 𝐸 = margin of error
  145. 145. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 145 𝑥 = 𝑥( ± 𝐸 calculations are approximate values for 95% level of confidence
  146. 146. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 146 𝐸 ≈ 1 𝑛 ⟹ 𝑛 ≈ 1 𝐸 1 calculations are approximate values for 95% level of confidence
  147. 147. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 147 calculations are approximate values for 95% level of confidence
  148. 148. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 148 𝑛2344 = corrected sample size 𝑛 = sample size 𝑁 = size of population calculations are approximate values for 95% level of confidence
  149. 149. Paul Marx | Principles of survey research 𝑛2344 = 𝑛 (1 + 𝑛 − 1 / 𝑁) Margin of Error Approach to Determining Sample Size 149 Margin of Error 1% calculations are approximate values for 95% level of confidence
  150. 150. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 150 calculations are approximate values for 95% level of confidence 𝑛2344 = 𝑛 (1 + 𝑛 − 1 / 𝑁) Margin of Error 5%
  151. 151. Paul Marx | Principles of survey research Margin of Error Approach to Determining Sample Size 151 calculations are approximate values for 95% level of confidence 𝑛2344 = 𝑛 (1 + 𝑛 − 1 / 𝑁) Margin of Error 10%
  152. 152. Paul Marx | Principles of survey research A Note on Confidence Interval 152 Confidence Interval & Level of Confidence A confidence interval estimate is an interval of numbers, along with a measure of the likelihood that the interval contains the unknown parameter. The level of confidence is the expected proportion of intervals that will contain the parameter if a large number of samples is maintained. . Suppose we're wondering what the average number of hours that people at Siemens spend working. We might take a sample of 30 individuals and find a sample mean of 7.5 hours. If we say that we're 95% confident that the real mean is somewhere between 7.2 and 7.8, we're saying that if we were to repeat this with new samples, and gave a margin of ±0.3 hours every time, our interval would contain the actual mean 95% of the time.
  153. 153. Paul Marx | Principles of survey research Confidence Interval, Margin of Error, and Sample Size 153 The higher the confidence we need, the wider the confidence interval and the greater the margin of error will be
  154. 154. Paul Marx | Principles of survey research Confidence Interval, Margin of Error, and Sample Size 154 The higher the confidence we need, the wider the confidence interval and the greater the margin of error will be smaller margins of error require larger samples higher levels of confidence require larger samples
  155. 155. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 155
  156. 156. Paul Marx | Principles of survey research Types of Statistical Data Analysis 156 Descriptive • Descriptive statistics provide simple summaries about the sample and about the observations that have been made. • Include the numbers, tables, charts, and graphs used to describe, organize, summarize, and present raw data. Inferential • Inferential statistics are techniques that allow making generalizations about a population based on random samples drawn from the population. • Allow assessing causality and quantifying relationships between variables.
  157. 157. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 157
  158. 158. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 158
  159. 159. Paul Marx | Principles of survey research blue red blue orange blue yellow green red pink blue green blue purple blue blue green yellow pink blue red pink green blue yellow green blue Frequency and Relative Frequency Tables 159 Original Data 𝑟𝑒𝑙𝑎𝑡𝑖𝑣𝑒 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 = 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 𝑠𝑢𝑚 𝑜𝑓 𝑎𝑙𝑙 𝑓𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑖𝑒𝑠 A frequency distribution lists each category of data and the number of occurrences for each category The relative frequency is the proportion (or percent) of observations within a category. A relative frequency distribution lists each category of data together with the relative frequency of each category. favorite color frequency blue 10 red 3 orange 1 yellow 3 green 5 pink 3 purple 1 favorite color relative frequency blue 10/26≈ 0.38 red 3/26≈ 0.12 orange 1/26≈ 0.04 yellow 3/26≈ 0.12 green 5/26≈ 0.19 pink 3/26≈ 0.12 purple 1/26≈ 0.04
  160. 160. Paul Marx | Principles of survey research favorite color relative frequency blue 10/26≈ 0.38 red 3/26≈ 0.12 orange 1/26≈ 0.04 yellow 3/26≈ 0.12 green 5/26≈ 0.19 pink 3/26≈ 0.12 purple 1/26≈ 0.04 favorite color frequency blue 10 red 3 orange 1 yellow 3 green 5 pink 3 purple 1 Bar Graphs 160 0 2 4 6 8 10 12 blue red orange yellow green pink purple FREQUENCY favorite color 0% 5% 10% 15% 20% 25% 30% 35% 40% blue red orange yellow green pink purple RELATIVE FREQUENCY favorite color Bar Graphs / Bar Charts 1. heights can be frequency or relative frequency 2. bars must not touch
  161. 161. Paul Marx | Principles of survey research Pie Charts 161 blue 37% red 12%orange 4% yellow 12% green 19% pink 12% purple 4% favorite color Pie Charts 1. should always include the relative frequency 2. also should include labels, either directly or as a legend
  162. 162. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 162
  163. 163. Paul Marx | Principles of survey research Exam Score Frequency 50–59 2 60–69 5 70–79 7 80–89 7 90–99 4 children frequency Relative frequency 1 3 3/26≈0.12 2 8 8/26≈0.31 3 10 10/26≈0.38 4 2 2/26≈0.08 5 3 3/26≈0.12 Tables 163 Original Data Original Data Sometimes there are too many values to make a row for each one. In that case, we'll need to group several values together. A discrete variable is a quantitative variable that has either a finite number of possible values or a countable number of values, i.e., 0, 1, 2, 3, ... 2 2 2 4 5 3 3 3 3 2 1 2 3 5 3 4 3 1 2 3 5 3 2 1 3 2 62 87 67 58 95 94 91 69 52 76 82 85 91 60 77 72 83 79 63 88 79 88 70 75 75 lower class limit upper class limit class width= 90-80 = 10
  164. 164. Paul Marx | Principles of survey research average commute frequency relative frequency 16–17.9 1 1/15≈0.07 18–19.9 2 2/15≈0.13 20–21.9 1 1/15≈0.07 22–23.9 6 6/15≈0.40 24–25.9 2 2/15≈0.13 26–27.9 1 1/15≈0.07 28–29.9 1 1/15≈0.07 30–31.9 1 1/15≈0.07 children frequency relative frequency 1 3 3/26≈0.12 2 8 8/26≈0.31 3 10 10/26≈0.38 4 2 2/26≈0.08 5 3 3/26≈0.12 Tables 164 0 2 4 6 8 10 12 1 2 3 4 5 FREQUENCY NUMBER OF CHILDREN IN FAMILY 0,00 0,10 0,20 0,30 0,40 0,50 1 2 3 4 5 RELATIVE FREQUENCY NUMBER OF CHILDREN IN FAMILY 0 1 2 3 4 5 6 7 16 18 20 22 24 26 28 30 32 FREQUENCY TIME (MINUTES) Average Daily Commute
  165. 165. Paul Marx | Principles of survey research Histogram 1. height of rectangles is the frequency or relative frequency of the class 2. widths of rectangles is the same and they touch each other 0 2 4 6 8 10 12 1 2 3 4 5 FREQUENCY NUMBER OF CHILDREN IN FAMILY 0,00 0,10 0,20 0,30 0,40 0,50 1 2 3 4 5 RELATIVE FREQUENCY NUMBER OF CHILDREN IN FAMILY 0 1 2 3 4 5 6 7 16 18 20 22 24 26 28 30 32 FREQUENCY TIME (MINUTES) Average Daily Commute Histogram 165 average commute frequency relative frequency 16–17.9 1 1/15≈0.07 18–19.9 2 2/15≈0.13 20–21.9 1 1/15≈0.07 22–23.9 6 6/15≈0.40 24–25.9 2 2/15≈0.13 26–27.9 1 1/15≈0.07 28–29.9 1 1/15≈0.07 30–31.9 1 1/15≈0.07
  166. 166. Paul Marx | Principles of survey research Frequency Polygon 166 0 1 2 3 4 5 6 7 16 18 20 22 24 26 28 30 32 FREQUENCY TIME (MINUTES) Average Daily Commute A frequency polygon is drawn by plotting a point above each class midpoint and connecting the points with a straight line. (Class midpoints are found by average successive lower class limits.) 16 21 26 31 0 1 2 3 4 5 6 7 16 18 20 22 24 26 28 30 32 FREQUENCY TIME (MINUTES) Average Daily Commute 0 1 2 3 4 5 6 7 15 17 19 21 23 25 27 29 31 33 FREQUENCY TIME (MINUTES) Average Daily Commute average commute frequency relative frequency 16–17.9 1 1/15≈0.07 18–19.9 2 2/15≈0.13 20–21.9 1 1/15≈0.07 22–23.9 6 6/15≈0.40 24–25.9 2 2/15≈0.13 26–27.9 1 1/15≈0.07 28–29.9 1 1/15≈0.07 30–31.9 1 1/15≈0.07
  167. 167. Paul Marx | Principles of survey research Cumulative Tables and Ogives 167 average commute relative frequency cumulative relative frequency 16–17.9 1/15≈ 0.07 1/15≈ 0.07 18–19.9 2/15≈ 0.13 2/15≈ 0.20 20–21.9 1/15≈ 0.07 1/15≈ 0.27 22–23.9 6/15≈ 0.40 6/15≈ 0.67 24–25.9 2/15≈ 0.13 2/15≈ 0.80 26–27.9 1/15≈ 0.07 1/15≈ 0.87 28–29.9 1/15≈ 0.07 1/15≈ 0.94 30–31.9 1/15≈ 0.07 1/15≈ 1.00 Cumulative tables show the sum of values up to and including that particular category. An ogive is a graph that represents the cumulative frequency or cumulative relative frequency for the class. average commute frequency cumulative frequency 16–17.9 1 1 18–19.9 2 3 20–21.9 1 4 22–23.9 6 10 24–25.9 2 12 26–27.9 1 13 28–29.9 1 14 30–31.9 1 15 0 0,2 0,4 0,6 0,8 1 1,2 17 19 21 23 25 27 29 31 33 Cumulative Relative Frequency Time (minutes) Average Daily Commute
  168. 168. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 168
  169. 169. Paul Marx | Principles of survey research Measures of Central Tendency 169 Mean 𝑥̅ = 𝑥H + 𝑥1 + ⋯ + 𝑥J 𝑛 = ∑ 𝑥L 𝑛 Sum of each item Sum of average items Mean is the “center of gravity” - like the balance point Advantages: • It works well for lists that are simply combined (added) together. • Easy to calculate: just add and divide. • It’s intuitive — it’s the number “in the middle”, pulled up by large values and brought down by smaller ones. Disadvantages: • The average can be skewed by outliers — it doesn’t deal well with wildly varying samples. • The average of 100, 200 and -300 is 0, which is misleading.
  170. 170. Paul Marx | Principles of survey research Measures of Central Tendency 170 Median Median is the item in the middle of a sorted list Advantages: • Handles outliers well — often the most accurate representation of a group • Splits data into two groups, each with the same number of items Disadvantages: • Can be harder to calculate: you need to sort the list first • Not as well-known; when you say “median”, people may think you mean “average” 50% below 50% above 𝑥M = N 𝑥(OPH)/1 1 2 𝑥O/1 + 𝑥O/1PH for odd n for even n
  171. 171. Paul Marx | Principles of survey research Measures of Central Tendency 171 Mode count item Mode is the most frequent observation of the variable Advantages: • Works well for exclusive voting situations (this choice or that one; no compromise), i.e., for nominal data • Gives a choice that the most people wanted (whereas the average can give a choice that nobody wanted). • Simple to understand Disadvantages: • Requires more effort to compute (have to tally up the votes) • “Winner takes all” — there’s no middle path The mode of is
  172. 172. Paul Marx | Principles of survey research Measures of Central Tendency: Using Mean and Median to Identify the Distribution Shape 172 symmetric mean and median approximately equal left-skewed median mean is “pulled” down right-skewed median mean is “pulled” up
  173. 173. Paul Marx | Principles of survey research Measures of Dispersion 173 𝜎1 = ∑ 𝑥L − 𝜇 1 𝑛 Population Variance Sample Variance 𝑠1 = ∑ 𝑥L − 𝑥̅ 1 𝑛 − 1 Variance is the average of the squared distance form the mean Heights of the 2008 US Men's Olympic Basketball Team
  174. 174. Paul Marx | Principles of survey research Mean acts as a balancing point. Hence, the average difference from the mean will equal zero. When calculating variance, all differences are squared, so that negative differences do not compensate positive differences. Measures of Dispersion 174 Sample Variance 𝑠1 = ∑ 𝑥L − 𝑥̅ 1 𝑛 − 1 Heights of the 2008 US Men's Olympic Basketball Team 𝑥̅ = 1.5 + 2.5 + 3.5 − 0.5 + 4.5 + 1.5 − 2.5 − 6.5 + 2.5 − 0.5 − 2.5 − 3.5 12 = 0 𝑠1 = 117 12 − 1 ≈ 10.6 Why Variance?
  175. 175. Paul Marx | Principles of survey research Which data set has a higher standard deviation? Measures of Dispersion 175 Standard Deviation 𝑠 = 𝑠1 Standard Deviation keeps the units of the original measure 𝜎 = 𝜎1 𝑠 = 10,6 ≈ 3.3 𝑠1 = 117 12 − 1 ≈ 10.6 square inches inches
  176. 176. Paul Marx | Principles of survey research Relationship between the Standard Deviation and the Shape of the Normal Distribution 176 99,7% of the data are within 3 standard deviations of the mean 95% within 2 standard deviations 68% within 1 standard deviation © Dan Kernler
  177. 177. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 177
  178. 178. Paul Marx | Principles of survey research Cross-Tabulations 178 Cross-Tabulations Cross-Tabulations are tables that reflect the joint distribution of two (or more) variables with a limited number of categories or distinct values. • help to understand how one variable (e.g., brand loyalty) relates to another variable (e.g., sex) • a cross-tabulation table contains a cell for every combination of categories of two (or more) variables Examples: • How many brand-loyal users are males? • Is product use (heavy users, medium users, light users, and non-users) related to outdoor activities (high, medium and low)? • Is familiarity with a new product related to age and education levels? • Is product ownership related to income (height, medium, and low)?
  179. 179. Paul Marx | Principles of survey research Cross-Tabulation 179 Education Own Expensive Automobile College Degree No College Degree yes 32 % 21 % no 68 % 79 % Column total 100 % 100 % Number of cases 250 750 Does education influence ownership of expensive automobiles? Ownership of Expensive Automobiles by Education Level
  180. 180. Paul Marx | Principles of survey research Cross-Tabulation 180 Sometimes introducing a third variable can reveal spurious relationship suppressed association no change in initial relationship
  181. 181. Paul Marx | Principles of survey research Cross-Tabulation 181 Does education influence ownership of expensive automobiles? Ownership of Expensive Automobiles by Education and Income Levels Low Income High Income Own Expensive Automobile College Degree No College Degree College Degree No College Degree yes 20 % 20 % 40 % 40 % no 80 % 80 % 60 % 60 % Column total 100 % 100 % 100 % 100 % Number of cases 100 700 150 50 Does it?
  182. 182. Paul Marx | Principles of survey research Cross-Tabulation 182 Does age influence desire to travel? Desire to Travel Abroad by Age Ages Desire to travel abroad Less than 45 45 or more yes 50 % 50 % no 50 % 50 % Column total 100 % 100 % Number of cases 500 500 Male Female Desire to travel abroad < 45 ≥ 45 < 45 ≥ 45 yes 60 % 40 % 35 % 65 % no 40 % 60 % 65 % 35 % Column total 100 % 100 % 100 % 100 % Number of cases 300 300 200 200 Desire to Travel Abroad by Age and Sex
  183. 183. Paul Marx | Principles of survey research Cross-Tabulation 183 Does family size influence frequency of eating in fast-food restaurants? Eating Frequency in Fast-Food Restaurants by Family Size Eat frequently in fast-food restaurants Family size Small Large yes 50 % 50 % no 50 % 50 % Column total 100 % 100 % Number of cases 500 500 Eat frequently in fast-food restaurants Low income High income Small Large Small Large yes 50 % 50 % 50 % 50 % no 50 % 50 % 50 % 50 % Column total 100 % 100 % 100 % 100 % Number of cases 250 250 250 250 Eating Frequency in Fast-Food Restaurants by Family Size and Income
  184. 184. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 184
  185. 185. Paul Marx | Principles of survey research 5.Data Analysis: A Concise Overview of Statistical Techniques 5.1 Descriptive Statistics: Some popular Displays of Data 5.1.1 Organizing Qualitative Data 5.1.2 Organizing Quantitative Data 5.1.3 Summarizing Data Numerically 5.1.4 Cross-Tabulations 5.2 Inferential Statistics: Can the Results Be Generalized to Population? 5.2.1 Hypotheses Testing 5.2.2 Strength of a Relationship in Cross-Tabulation 5.2.3 Describing the Relationship between Two (Ratio Scaled) Variables 185
  186. 186. Paul Marx | Principles of survey research Hypothesis Testing 186 Hypothesis Testing Hypothesis Testing is a five-step procedure using sample evidence and probability theory to determine whether the hypothesis is a reasonable statement. In other words, it is a method to prove whether or not the results obtained on a randomly drawn sample are projectable to the whole population. Procedure: 1. State null and alternative hypothesis 2. Select a level of significance 3. Identify the test statistic 4. Formulate a decision rule 5. Take a sample, arrive at a decision "People are 'erroneously confident' in their knowledge and underestimate the odds that their information or beliefs will be proved wrong. They tend to seek additional information in ways that confirm what they already believed." Max Bazerman
  187. 187. Paul Marx | Principles of survey research Hypothesis Testing 187 Sex Internet usage Male Female Row total light 5 10 15 heavy 10 5 15 Column total 15 15 n=30 Sex and Internet Usage Based on this sample: Q: Are there really more heavy internet users among males than among females in the general population?
  188. 188. Paul Marx | Principles of survey research Hypothesis Testing 188 Step 1: State null and alternative hypothesis A null hypothesis ( 𝑯 𝟎) is a statement of status quo, one of no difference or no effect. An alternative hypothesis ( 𝑯 𝟏) is one in which some difference or effect is expected. 𝑯 𝟎: There is no difference between males and females w.r.t. internet usage. 𝑯 𝟏: Males and females expose different internet usage behavior. 𝐼𝑈` = 𝐼𝑈a 𝐼𝑈` ≠ 𝐼𝑈a
  189. 189. Paul Marx | Principles of survey research Hypothesis Testing 189 Step 2: Select a level of significance Significance ( 𝜶) – probability of rejecting a true null hypothesis. 𝜷 – probability of accepting a false null hypothesis. Null hypothesis (𝐻0) is true Null hypothesis (𝐻0) is false Reject null hypothesis Type I error False positive Correct outcome True positive Fail to reject null hypothesis Correct outcome True negative Type II error False negative 𝛽 (1 − 𝛽) – power of test 𝛼 – significance
  190. 190. Paul Marx | Principles of survey research Null hypothesis (𝐻0) is true Null hypothesis (𝐻0) is false Reject null hypothesis Type I error False positive Correct outcome True positive Fail to reject null hypothesis Correct outcome True negative Type II error False negative Hypothesis Testing 190 acquit a criminal convict an innocent Analogy: innocence in a criminal trial 𝐻0: the defendant is innocent Step 2: Select a level of significance Significance ( 𝜶) – probability of rejecting a true null hypothesis. 𝜷 – probability of accepting a false null hypothesis.
  191. 191. Paul Marx | Principles of survey research Null hypothesis (𝐻0) is true Null hypothesis (𝐻0) is false Reject null hypothesis Type I error False positive Correct outcome True positive Fail to reject null hypothesis Correct outcome True negative Type II error False negative Hypothesis Testing 191 you continue your business near the bush but a lion is there there is no lion but you run away Analogy: Rustle in the bush – is it a lion? 𝐻0: there is no lion in the bush Step 2: Select a level of significance Significance ( 𝜶) – probability of rejecting a true null hypothesis. 𝜷 – probability of accepting a false null hypothesis.
  192. 192. Paul Marx | Principles of survey research Hypothesis Testing 192 Levels of significance in marketing research 𝛼 – level of significance (1 − 𝛼) – level of confidence 0.01 (1%) 0.05 (5%) 0.99 (99%) 0.95 (95%) Step 2: Select a level of significance Significance ( 𝜶) – probability of rejecting a true null hypothesis. 𝜷 – probability of accepting a false null hypothesis.
  193. 193. Paul Marx | Principles of survey research Hypothesis Testing 193 Step 3: Identify the test statistic Sample Application Level of scaling Test/Comments One Sample Distributions Non-metric Kolmogorow-Smirnow and χ2 test for goodness of fit; Runs test for randomness; Binomial test for goodness of fit of dichotomous variables Means Metric t test, if variance is unknown z test, if variance is known Proportions Metric z test Two Independent Samples Distributions Non-metric Kolmogorow-Smirnow two-sample test for equality of two distributions Means Metric Two-group t test F test for equality of variances Proportions Metric
Non-metric z test χ2 test Ranking/Medians Non-metric Mann-Whitney U test is more powerful than the median test Paired Samples Means Metric paired t test Proportions Non-metric McNemar test for binary variables, χ2 test Ranking/Medians Non-metric Wilcoxon matched-pairs ranked-signs test is more powerful than the sign test
  194. 194. Paul Marx | Principles of survey research Hypothesis Testing 194 Step 3: Identify the test statistic Sample Application Level of scaling Test/Comments One Sample Distributions Non-metric Kolmogorow-Smirnow and χ2 test for goodness of fit; Runs test for randomness; Binomial test for goodness of fit of dichotomous variables Means Metric t test, if variance is unknown z test, if variance is known Proportions Metric z test Two Independent Samples Distributions Non-metric K-S two-sample test for equality of two distributions Means Metric Two-group t test F test for equality of variances Proportions Metric
Non-metric z test χ2 test Ranking/Medians Non-metric Mann-Whitney U test is more powerful than the median test Paired Samples Means Metric paired t test Proportions Non-metric McNemar test for binary variables, χ2 test Ranking/Medians Non-metric Wilcoxon matched-pairs ranked-signs test is more powerful than the sign test ! In our example, we deal with one-sample distribution of a non-metric variable (light or heavy internet usage)
  195. 195. Paul Marx | Principles of survey research Hypothesis Testing 195 Step 3: Identify the test statistic χ2 (chi-square) statistic for goodness of fit is used to test the statistical significance of the observed association in a cross-tabulation 𝐻0: There is no association between the variables χ2 (chi-square) tests the equality of frequency distributions. Which distributions/frequencies should we test? 𝑓 𝑒 – cell frequencies that would be expected if no association were present between the variables 𝑓 𝑜 – actual observed cell frequencies
  196. 196. Paul Marx | Principles of survey research Hypothesis Testing 196 Step 3: Identify the test statistic 𝑓h = 𝑛4 𝑛2 𝑛 𝑛4 – total number in the row 𝑛2 – total number in the column 𝑛 – total sample size 𝑓hi,i = 15 j 15 30 = 7,5 𝑓hi,k = 15 j 15 30 = 7,5 𝑓hk,i = 15 j 15 30 = 7,5 𝑓hk,k = 15 j 15 30 = 7,5 𝑓 𝑒 – cell frequencies that would be expected if no association were present between the variables 𝑓 𝑜 – actual observed cell frequencies
  197. 197. Paul Marx | Principles of survey research Hypothesis Testing 197 Step 3: Identify the test statistic In our example: 𝜒1 = (mno.m)k o.m + (Hpno.m)k o.m + (Hpno.m)k o.m + (mno.m)k o.m = 0.833 + 0.833 + 0.833 + 0.833 = 3.333 𝜒1 = r (𝑓3 − 𝑓h)1 𝑓hall cells 𝑓 𝑒 – cell frequencies that would be expected if no association were present between the variables 𝑓 𝑜 – actual observed cell frequencies
  198. 198. Paul Marx | Principles of survey research Hypothesis Testing 198 Step 4: Formulate a decision rule 𝑻𝑺 𝒄𝒂𝒍 – observed value of the test statistic. 𝑻𝑺 𝒄𝒓 – critical value of the test statistic for a given significance level. If probability of 𝑻𝑺 𝒄𝒂𝒍 < significance level (𝜶), then reject 𝑯 𝟎. or If 𝑻𝑺 𝒄𝒂𝒍 > 𝑻𝑺 𝒄𝒓 , then reject 𝑯 𝟎.
  199. 199. Paul Marx | Principles of survey research Hypothesis Testing 199 Step 4: Formulate a decision rule If probability of 𝑻𝑺 𝒄𝒂𝒍 < significance level (𝜶), then reject 𝑯 𝟎. or If 𝑻𝑺 𝒄𝒂𝒍 > 𝑻𝑺 𝒄𝒓 , then reject 𝑯 𝟎. 𝑑𝑓 Table of critical values of χ2 for different levels of significance 𝛼 𝑑𝑓 – degrees of freedom 𝑟 – number of rows 𝑐 – number of columns 𝑑𝑓 = 𝑟 − 1 𝑐 − 1 𝑑𝑓 = 2 − 1 2 − 1 = 1 𝜒2|} 1 = 3.333 𝜒24 1 = 3.841 3.333 < 3.841 𝜒2|} 1 < 𝜒24 1 𝐻0 cannot be rejected
  200. 200. Paul Marx | Principles of survey research Hypothesis Testing 200 Step 5: Arrive at a decision Is the evidence there? What are the consequences? • 𝑯 𝟎 of no association cannot be rejected • Association is not statistically significant at the .05 level • The findings from the sample cannot be generalized to population
  201. 201. Paul Marx | Principles of survey research Hypothesis Testing 201 Sex Internet usage Male Female Row total light 5 10 15 heavy 10 5 15 Column total 15 15 n=30 Sex and Internet Usage Based on this sample: Q: Are there really more heavy internet users among males than among females in the general population? A: The sample doesn’t provide such evidence. If the sample was chosen and drawn appropriately, then we can state that there is no such relationship in the population at the 95% confidence level. Otherwise - we don’t know.

×