Existing and new approaches for analysing data from Check All That Apply questions
•
1 like•665 views
Check-All-That-Apply (CATA) questions are increasingly being incorporated into consumer tests because they provide a simple mechanism for consumers to communicate their perceptions of products being evaluated. We review existing and propose new approaches for analysing data obtained from such a study. Contingency tables are well known, and can be pictured using mosaic plots. Correspondence analysis (CA) using the χ2 distance provides dimensionality reduction, but Hellinger's distance is often preferred where rarely cited attributes skew results. Word clouds can be used to determine citation frequency for responses that might be entered in open comment format by consumers (e.g. upon checking "other" in a CATA question). Cochran's Q test provides a univariate test for differences between 3 or more products, and the sign test can be used to assess pairwise differences. To our knowledge no omnibus hypothesis test is available for assessing global differences. We propose such a test, based on randomization and Cochran's Q statistics, in which the null distribution is formed from data re-randomizations. Multidimensional alignment (MDA) is suggested to investigate the relationship between products and CATA attributes. The φ-coefficients, proposed to understand relationships between CATA attributes, are readily visualized using MDS. Consumers can be asked to evaluate an ideal product, and the gaps between the real and ideal products can inform product improvements. Penalty and penalty-lift analyses can reveal (positive and negative) hedonic drivers. Methods are illustrated by means of CATA study on whole grain breads.
Recommended
More Related Content
Similar to Existing and new approaches for analysing data from Check All That Apply questions
Similar to Existing and new approaches for analysing data from Check All That Apply questions (20)
More from Compusense Inc.
More from Compusense Inc. (13)
Recently uploaded
Recently uploaded (20)
Existing and new approaches for analysing data from Check All That Apply questions
- 1. Existing and new approaches for the analysis of CATA data Michael Meyners Procter & Gamble B. Thomas Carr Carr Consulting John C. Castura Compusense Inc.
- 2. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 3. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 4. Are 2 products different? Sign Test (exact) McNemar’s Test (approximation) Selected Not Selected Selected a b Not Selected c d Product B ProductA
- 5. Are 3 (or more) products different? Cochran’s Q Test (approximation) extends McNemar’s Test
- 6. Testing strategy Attributes Products Test 1 All 1 2 Cochran’s Q Test Sign Test Significant?
- 7. Testing strategy Attributes Products Test All All 1 All 1 2 ? Global Test ? Cochran’s Q Test Sign Test Significant? Significant?
- 8. Global Test Test Statistic: Sum of 𝑄 statistics Approximate Test Chi-square distribution formed using property of additivity Problem: null distribution incorrect (chi-square distributions are not independent)
- 9. Global Test Solution: Randomization test (quasi-exact): null distribution is valid because it is based on repeated and appropriate reshufflings of observed data
- 10. Testing strategy Attributes Products Test All All 1 All 1 2 Randomization Test Cochran’s Q Test Sign Test Significant? Significant?
- 11. With CATA data, can we… know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts?
- 12. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 13. 1 2 3 … Ideal Total Fresh 89 70 71 … 142 642 Warm 6 7 4 … 90 135 Crusty 24 4 32 … 47 193 Soft 76 95 75 93 570 Sweet 102 18 30 85 466 … … … … … ... … Total 2169 1144 1539 … 2579 Contingency table
- 14. Bar charts
- 15. Problem: This is a mess. We need a visual summary of this data. Bar charts
- 16. Correspondence Analysis Reduces dimensionality of contingency tables (Think: PCA for ordinary data) Attributes Products
- 17. Correspondence Analysis Could use… Chi-square distances Sensitive to attributes with low counts Hellinger distances Low counts not a problem
- 18. Correspondence Analysis Sweet 2 4 3 1 5 6 Molasses Exciting Tempting Satisfying Tasteful FullnessComfort Dense Firm Moist Warm Homemade Fresh Chewy Traditional SoftSpringy Wheat Brown Healthy Nutritious Crusty Texture Coarse Sesame RusticNutty Crunchy Grainy Seeds Component 1 (79.2%) Component2(11.9%)
- 19. Component 1 (79.2%) 23 1 5 6 Traditional SoftSpringy Brown Nutty Crunchy Grainy Seeds 4
- 20. Correspondence Analysis Can only look at 2 (or 3) dimensions at once Problem: True relationships between products and attributes might be obscured.
- 21. Multidimensional Alignment (MDA) Solution: Look at each product individually.
- 24. With CATA data, can we… know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts?
- 25. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 26. φ-coefficient Measures correlation between 2 binary attributes
- 27. Classical MDS on matrix of 𝟏 − 𝝋 distances Fresh WarmCrusty Soft Moist Nutty Tempting Sweet Dense Texture Wheat Exciting Crunchy Homemade Seeds Satisfying Coarse Nutritious Molasses Tasteful Rustic Traditional Springy Sesame Firm Fullness Healthy Grainy Comfort Brown Chewy
- 28. Classical MDS on matrix of 𝟏 − 𝝋 distances Fresh WarmCrusty Soft Moist Nutty Tempting Sweet Dense Texture Wheat Exciting Crunchy Homemade Seeds Satisfying Coarse Nutritious Molasses Tasteful Rustic Traditional Springy Sesame Firm Fullness Healthy Grainy Comfort Brown Chewy
- 29. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 30. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 31. Penalty-Lift Analysis Problem: What is the relationship between what consumers perceive and their hedonic response? Solution: Look at elicitations of attributes and association with liking
- 32. change in liking (attribute elicited vs. not elicited)
- 33. change in liking (attribute elicited vs. not elicited)
- 34. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 35. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we…
- 36. Comparison with Ideal Problem: How to modify a product to delight consumers? Solution 1: Close gap between product and ideal (accounting for natural variability!)
- 37. Difference in elicitation rates Product – Ideal
- 38. Difference in elicitation rates Product – Ideal
- 39. Penalty Analysis Problem: How to modify a product to delight consumers? Solution 2: Focus on the most relevant attributes for consumers’ hedonic responses
- 40. % consumers Higher Relevance for Many Consumers +2 +1 0 10 30 504020 Penalty analysis based on “must have” attributes Liking Fresh Warm Crusty Soft Moist Chewy Nutty Tempting Sweet Grainy Dense Texture Healthy Exciting Crunchy Homemade Brown Seeds Satisfying Coarse Fullness Nutritious Molasses Tasteful RusticTraditional Springy Sesame Wheat Comfort
- 41. know if products are different? characterize products? know how attributes are correlated? know what drives consumer liking? focus reformulation efforts? With CATA data, can we… Yes!
- 42. Thank you for your attention! For more information see the following publication and references therein: Meyners, M., Castura, J.C., Carr, B.T. (2013). Existing and new approaches for the analysis of CATA data. Food Quality and Preference, 30, 309-319. B. Thomas Carr Carr Consulting Michael Meyners Procter & Gamble John C. Castura Compusense Inc.