Concept calibration: ESOMAR Innovate conference 2008

731
-1

Published on

How can scores from a concept tested in Germany be reliably compared to those from the same concept tested in China? Measured and true concept scores can vary widely between countries due to e.g. cultural response behavior, response styles, time of measurement or consumer innovation profile. Based on a 14-country study covering Europe, the Americas and Asia-Pacific, the authors have developed a framework for assessing ‘pure’ concept performance independent of contextual or biasing factors, providing an alternative approach to the often criticized traditional benchmarking of concepts.

Published in: Business, Design
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
731
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Concept calibration: ESOMAR Innovate conference 2008

    1. 1. Beyond benchmarking Filip De Boeck Kristof De Wulf
    2. 2. Prediction is very difficult, especially if it’s about the future (Niels Bohr)      “ Prediction is very difficult, especially if it's about the future. ”   Niels Bohr
    3. 5. A structured research process to guide decisions
    4. 6. Type II error Type I error NO GO GO NO GO GO Appropriate action Action taken A product / concept that has the potential to generate strong trial and should be proceeded through the innovation funnel. A product / concept that has the potential to generate strong trial, BUT requires some adaptations before being considered for market launch. A product/concept that will struggle to maintain distribution .
    5. 7. Benchmarking in order to reduce type I and II error
    6. 8. Current benchmarking practices <ul><li>Mainly based on one variable only </li></ul><ul><ul><li>5-point Likert scale: definitely buy, probably buy, not sure, probably not buy and definitely not buy </li></ul></ul><ul><ul><li>Variants / computations : TOP 1, TOP 2, weighted score </li></ul></ul><ul><li>Benchmarking this information with historically tested concepts in relevant markets & categories </li></ul><ul><li>And using action standards per market & category to determine whether a concept is good, moderate or bad </li></ul>
    7. 9. Shortcomings of benchmarking Reflections of the past
    8. 10. Shortcomings of benchmarking Research design bias
    9. 11. Shortcomings of benchmarking Sampling distortions
    10. 12. Shortcomings of benchmarking TRIAL 10,0 15,0 20,0 25,0 30,0 35,0 40,0 45,0 50,0 30% 40% 50% 60% 70% 80% 90% 100% UNIQUENESS Comparability of data across markets Australia Brazil Canada China Germany France Hungary Italy Holland Russia Spain UK US Belgium bias / noise true difference Difference between markets (better fit of concept with local habits, maturity of the category,...) (response style, sampling distortion, ...)
    11. 13. Study design <ul><li>Online survey </li></ul><ul><ul><li>14 markets (Belgium, the Netherlands, France, UK, Germany, Spain, Italy, Russia, Canada, Australia, China, Brazil and Hungary) </li></ul></ul><ul><li>95 product ideas </li></ul><ul><li>Measurement </li></ul><ul><ul><li>Representative household sample – N = 150 / concept </li></ul></ul><ul><ul><li>Random assignment to one category (non-rejection of the category) </li></ul></ul><ul><ul><li>10 random product ideas – product evaluation </li></ul></ul><ul><ul><li>Profile : early adoptership, response style (Greenleaf), demographics </li></ul></ul>
    12. 14. Innovation Potential Index (IPI) <ul><li>Reflects the individual innovation potential of a concept </li></ul><ul><li>Reflects how well a specific concept performs for a consumer against that consumer’s natural tendency to adopt a new product within the product category </li></ul>
    13. 15. Innovation Potential Index (IPI) <ul><li>Reflects the individual innovation potential of a concept </li></ul><ul><li>Reflects how well a specific concept performs for a consumer against that consumer’s natural tendency to adopt a new product within the product category </li></ul>
    14. 16. Innovation Potential Index (IPI) Anna Maarten Ravi Juan Dave Buying intent new concept Natural tendency to adopt Innovation Potential Index + + + + + = = = = =
    15. 17. Market Potential Index (MPI) <ul><li>Reflects true market potential of a certain concept in a country </li></ul><ul><li>Indicates how well a new concept performs against the average level of buying intentions across concepts in a specific market </li></ul>
    16. 18. Key research propositions <ul><li>IPI is free from response style bias </li></ul><ul><li>IPI is independent from personal characteristics </li></ul><ul><li>MPI leads to more spread in cross-country concept performance </li></ul><ul><li>Average MPI across concepts is equal between countries </li></ul><ul><li>MPI is strongly correlated with unpriced buying intent scores </li></ul>    
    17. 19. InSites International Innovation Matrix Assess cross-country potential of a concept
    18. 20. InSites International Innovation Matrix Assess cross-country potential of a concept Fight for priority Ride the tube Bail out Assess wave quality
    19. 21. Case in point: potato concept
    20. 22. Case in point: potato concept Extrinsic winners True winners Low downs Intrinsic winners Market Potential Index Innovation Potential Index From ‘go’ to ‘no go’ From ‘no go’ to ‘go’ Australia Brazil Canada China Germany France Hungary Italy Holland Russia Spain UK US Belgium
    21. 23. Case in point: sauce concept
    22. 24. Case in point: sauce concept Extrinsic winners True winners Low downs Intrinsic winners Market Potential Index Innovation Potential Index From ‘go’ to ‘no go’ From ‘no go’ to ‘go’ Australia Brazil Canada China Germany France Hungary Italy Holland Russia Spain UK US Belgium
    23. 25. Implications <ul><li>Benchmarking is possible across countries, eliminating the need for country-specific benchmarking and related action standards. The InSites International Innovation Matrix can directly be used to assess concept performance in new emerging markets for which no benchmarking information is available. </li></ul><ul><li>Personal characteristics which are not accounted for in screeners, quota and/or the benchmark database do not have an impact on IPI scores. IPI scores can be compared between innovators and laggards, males and females, brand users and non users,… </li></ul><ul><li>Go or no go recommendations for our concept example were different in 16% of the cases. Given the large investments related to new product introductions, applying the InSites International Innovation Matrix methodology can hopefully lower the rate of failure of new products. </li></ul>
    24. 26. Some limitations <ul><li>Both IPI and MPI are defined within the boundaries of a particular product category: more tests need to be conducted on the impact of the definition of the product category on outcomes (e.g. from broad to narrow). </li></ul><ul><li>Given the cross-sectional design of the study, we cannot derive whether IPI and MPI are stable over time . A longitudinal design would allow to detect this. </li></ul><ul><li>Although we would alter our go-no go decision in 16% of the cases, we are not 100% sure whether this would actually lead to a better decision. Linking our results to after-launch in-market results could provide more guidance in the future. </li></ul>
    25. 27. Thanks for listening. Looking forward to your ideas!

    ×