“A new customer experience measurement model – a meta analytical review of findings over the period 2002 to 2009,” adré schreuder, consulta research, south africa

  • 5,439 views
Uploaded on

“A new customer experience measurement model – a meta analytical review of findings over the period 2002 to 2009,” by andré schreuder, consulta research, south africa. …

“A new customer experience measurement model – a meta analytical review of findings over the period 2002 to 2009,” by andré schreuder, consulta research, south africa.

Great integration of current knowledge

More in: Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
5,439
On Slideshare
0
From Embeds
0
Number of Embeds
1

Actions

Shares
Downloads
329
Comments
0
Likes
6

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A new Customer Experience Measurement Model – A Meta Analytical Review of Findings over the period 2002 to 2009 Presented by Prof Adré Schreuder MD of Consulta Research & Extra-ordinary Professor of Marketing Research – University of Pretoria, South Africa In partnership with: 19th Annual Frontiers in Service Conference 2010 10-13 June 2010 - Karlstad, Sweden
  • 2. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Dangers of Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 2
  • 3. Background & Rationale for Research Slide 3
  • 4. Terminology Confusion Source: Created by Adré Schreuder – reference: < http://www.wordle.net/show/wrdl/1954142/Customer_experience > Slide 4
  • 5. A Historic Overview CUSTOMER SATISFACTION Relationship Customer Product Quality Service Quality Quality Era Experience Era Era (1950’s) Era (1984) (1995) (2003) TQM SERVQUAL CRM CEM • TQM of Edwards • The Nordic approach (Grönroos 1984: Technical/Functional Deming - Zero Model, Lethinen & Lethinen 1988 : Technical, Corporate, Defect, Six Sigma Interactive) • The North American Debate (PZB 1985: SERVQUAL (Gap-based measure, Familiar five quality dimensions, Cronin & Taylor 1992: SERVPERF - Performance only measure, Brown Churchill & Peter 1993: Better/worse than expected scale, Teas 1993: Evaluated Performance Model = gap between perceived performance & ideal amount of feature) • Jagdish Sheth introduced Relationship Management in mid 90’s • Growth of CRM-systems and popularity • NPS introduced by Reichheld in 2003 – CEM era is born Slide 5
  • 6. Customer satisfaction: Contrasting academic and consumers’ interpretations Satisfaction defined – Derived from Latin “satis” = enough & “facere” (faction) = to do/to make – Early interpretation and use of the word mostly focused on “some sort of release from wrong doing” - later “release from uncertainty” • At least two basic approached in defining the concept: – CS viewed as an outcome of a consumption activity – CS viewed as a process • Most widely adopted description = evaluation between what was received and what was expected Source: Parker, C & Mathews, B.P. Marketing Intelligence & Planning, 19/1 2001 (pp 38-44) Slide 6
  • 7. Customer satisfaction: CS viewed as an outcome - Focus on the nature (not cause) of satisfaction: • Emotion - satisfaction is the surprise element of product acquisition and/or consumption experiences, or an affective response to a specific consumption experience • Fulfilment - motivation theories state that either people are driven by the desire to satisfy their needs or achieving specific goals. • State - Oliver’s (1989) framework of four satisfaction states, where satisfaction is related to reinforcement and arousal. – Low arousal = “satisfaction-as-contentment” – High arousal = “satisfaction as surprise” (positive / delight or negative / shock) – Positive reinforcement = “satisfaction-as-pleasure” – Negative reinforcement = “satisfaction-as-relief” Source: Parker, C & Mathews, B.P. Marketing Intelligence & Planning, 19/1 2001 (pp 38-44) Slide 7
  • 8. Customer satisfaction: CS viewed as a process • Concentrate on the antecedents to satisfaction rather than satisfaction itself. (Origins in discrepancy theory - (Porter, 1961) and Contrast Theory (Cardozo, 1965); • Most common interpretation = a feeling which results from a process of evaluating what was received against that expected, the purchase decision itself and/or the fulfillment of needs/wants. • Most “well-known’’ descendent of the discrepancy theories is the expectation disconfirmation paradigm (Oliver, 1977, 1981). Source: Parker, C & Mathews, B.P. Marketing Intelligence & Planning, 19/1 2001 (pp 38-44) Slide 8
  • 9. Customer Experience – the new “Customer Satisfaction”? • “Yet despite the recognition of the importance of customer experience by practitioners, the academic marketing literature investigating this topic has been limited. • Publications on customer experience are mainly found in practitioner-oriented journals or management books … tend to focus more on managerial actions and outcomes… • The literature in marketing, retailing and service management historically has NOT considered customer experience as a separate construct. Instead researchers have focused on measuring customer satisfaction and service quality.” Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A. Schlesinger (2009), “Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of Retailing, 85 (1), 31–41. Slide 9
  • 10. Customer Experience – the new “Customer Satisfaction”? • One reason for the apparently weak observed link between satisfaction and future behaviour may lie in the role of emotions • Previously studies emphasised cognitive aspects of satisfaction – growing body of evidence that affective measures of satisfaction (which incorporate emotions) may be a better predictor of behaviour • As a cognitive measure, satisfaction is more likely to be distorted over time than a measure that incorporates an affective component (emotions are more deep-seated & more stable over time) • Satisfaction should thus include a combination of an evaluative (cognitive) and emotion-based (affective) response to a service encounter Source: Koenig-Lewis, N. and Palmer, A. "Experiential values over time – a comparison of measures of satisfaction and emotion," Journal of Marketing Management (24:1-2), 2008, pp. 69-85. Slide 10
  • 11. Construct definition of “Customer Experience” • The customer experience construct is holistic in nature and involves the customer’s cognitive, affective, emotional, social and physical responses to the retailer. • This experience is created by: – controllable elements - service interface, retail atmosphere, assortment, price, – uncontrollable elements - influence of others, purpose of shopping • Customer experience encompasses the total experience, including the search, purchase, consumption, and after-sale phases of the experience, and may involve multiple retail channels. • Three major focus areas: – cognitive evaluations (i.e., functional values) – affective (emotional) responses – social and physical components Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A. Schlesinger (2009), “Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of Retailing, 85 (1), 31–41. Slide 11
  • 12. Putting Customer Experience into Perspective • The term Customer Experience Management is used within the broader context of Customer Relationship Management (CRM) – clearly seen in the view of Kirkby, Wecksell & Janowski (2003) when they say: “CEM is part of customer relationship management (CRM) and the natural extension of building brand awareness. • Where brand gives the promise, CEM is the physical delivery of that promise and is vital in an economy where a brand is increasingly built on value delivered rather than product Illustration Copyright – Consulta 2010 features”. Slide 12
  • 13. Putting Customer Experience in Perspective Slide 13
  • 14. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Dangers of Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 14
  • 15. Previous Research & Literature Review • Collection of previous research and literature regarding Customer Experience measurement are presented and discussed under the following topics: – Multi-attribute measures such as: • SERVQUAL, • ASCI & • Others – Effort Score & ERIC – Net Measures such as: • The Net Promoter Score from Fred Reichheld & Bain Company • Secure Customer Index from Burke Slide 15
  • 16. Customer satisfaction and company profitability: The Service-Profit Chain Operating Strategy & Service Delivery System Revenue Employee Growth Retention Internal External Employee Customer Customer Service Service Satisfaction Satisfaction Loyalty Quality Value Employee Productivity Profitability • Workplace Design 3Rs (>Market Share) • Job Design • Retention, Service Concept: • Repeat Business • Employee Selection & Development (skills Results for Customer & empowerment drives good feelings • Referrals towards the firm) • Employee Rewards & Recognition Service designed & delivered to meet targeted customer’s • Tools for Serving Customers needs Adapted from: Heskett, Jones, Loveman, Sasser & Schlesinger (HBR – 1994, HBR July/Aug 2008, p.120) Slide 16
  • 17. The GAP never mentioned … CEM = delivering what our Expectations customers expect us to – and a little bit more ‐, making them feel great at Gap 5 every “moment of truth”, Perceptions CEM – The “Missing Gap” Delivery Gap 4 Marketing & Interface Communication Gap 3 Experience Standards Gap 2 Adapted from original Gaps- Management Gap 1 Model of Parasuraman, understanding of Zeithaml & Berry expectations Illustration Copyright – Consulta 2010 Slide 17
  • 18. Conceptual Model of Customer Experience Creation Social Environment: Reference group, tribes, co-destruction, service staff Situational Moderators: Service Interface: Type of store, location, Service person, technology, co-creation/customisation culture, economic climate, season, competition Retail Atmosphere: Design, scents, temperature, ambient noise, music CEM Strategy Customer Experience Assortment: (t): Variety, uniqueness, quality Cognitive, affective, social, physical Price: Loyalty programs, promotions, rewards Consumer Customer experiences in alternative Moderators: channels Goals: experiential Task orientation, socio- demographics, consumer Retail Brand attitudes (price sensitivity, involvement) CUSTOMER EXPERIENCE (t – 1) Source: Verhoef, Peter C., Katherine N. Lemon, A. Parasuraman, Anne Roggeveen, Michael Tsiros and Leonard A. Schlesinger (2009), “Customer Experience Creation: Determinants, Dynamics and Management Strategies,” Journal of Retailing, 85 (1), 31–41. Slide 18
  • 19. Effort Score – worth the effort? Research conducted by Customer Contact Council of the Corporate Executive Board High Council Conclusion Comments: Better suited for service Effort channel. Better financial • Directly contrasting scientific predictor & best indicator of proof of ACSI (American), SCSI loyalty (Sweden) Council Conclusion • No scientific foundation Repurchase NPS® • Irresponsible to “recommend” Predictive “Inadequate measure” in the Power* of service channel: • Question inherently positive members against (only likelihood to recommend • Effort-score purely developed in – not criticize) • Captures company-level Contact centre environment sentiment (incl brand, product, pricing) • No published proof of scientific reliability & validity Council Conclusion • Scale is reverse scored – South CSAT Popular, widely used BUT “not African research shows low sufficient in predicting financial outcomes … de- reliability & poor predictive emphasize its use in strategic properties to the contrary Low decisions” Low High Predictive Power* for Increased Spend Power* - Linear regression coefficients regressed against Likelihood to Repurchase & Increase Spend Slide 19
  • 20. ERIC™ – Empathy Rating Index • The ERIC instrument consists of 29 empathy questions measured on a 10-point rating scale and 11 call process questions that are related to how the calls are processed • The trained researchers (mystery callers) then make 40 unscripted(?) calls over three weeks to each company and complete an online questionnaire • The study sample was limited to 28 companies in which ROCE and ERIC ratings were both available. Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of the empathy rating index (ERIC) in UK call centres," Journal of Database Marketing & Customer Strategy Management (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper 2005 < http://www.empathy.co.uk/ > Slide 20
  • 21. ERIC – Testing the claims Comments: Claimed at 2008 CS Conference: • No proven scientific grounding “At Last –a proven link between a service • Non rated Journal, 6 rated references related measure and profitability” used • Questionable statistics & sample • No longitudinal data or reference to time • Methodology basically mystery caller • Psychometric properties of scale – no scientific grounding • Mixed construct in scale (15 constructs across 33 statements • Of 5 attributes only one (Empathy) is an interval scale, all other “Yes/no” or numerical (number of calls) • Claimed at 2008 CS Conference = False claim Source: Lywood, J., Stone, M. and Ekinci, Y. "Customer experience and profitability: An application of the empathy rating index (ERIC) in UK call centres," Journal of Database Marketing & Customer Strategy Management (16), 2009, pp. 207-214. & Lywood, J., Stone, M. and Hackett, D. Eric Methodology Whitepaper 2005 < http://www.empathy.co.uk/ > Slide 21
  • 22. Net Promoter Score – single net measure • A simple recommend question measured on 0 to 10 scale of likelihood to recommend “How likely is it that you would recommend (brand or company X) to a friend or colleague?” • Net Promoter score is calculated by taking the percentage of “promoters” (9-10 rating; extremely likely) and the percentage of “detractors” (0-6 rating; extremely unlikely) NPS = % of Promoters minus % of Detractors • Companies with scores above 75% have world-class loyalty and word-of-mouth, which will correlate with a firm’s growth1 1Reichheld, F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 2003 Slide 22
  • 23. Net Promoter Score – single net measure Positive Negative • NPS adopted by executives: • Little scientific research linking • Swift to survey recommend intentions to actual • Simple to understand and intentions2 communicate • Morgan and Rego (2006) assessed • Top-of-house dashboard metric six different metrics over a seven • Reichheld (2003): NPS is a more year period and found: “…recent accurate predictor of sales growth prescriptions to focus customer than the elaborate American feedback systems & metrics solely Consumer Satisfaction Index1 on customers‟ recommendation • General Electrics CEO: “This is the intentions and behaviours are best customer satisfaction metric misguided”3 I‟ve seen” 1Reichheld,F. (2003). The One Number You Need to Grow. Harvard Business Review, Dec 2003 2Keiningham, T. et al. (2007). The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet, Managing Service Quality 17(4), 361-384. 3Morgan, N. & Rego, L. (2006). The Value of Different Customer Satisfaction and Loyalty Metrics in Predicting Business Performance. Marketing Science 25(5), Sep – Oct. Slide 23
  • 24. Testing the Net Promoter Score® claims • Contrary to Reichheld’s assertions, the results indicate that recommend intention alone will not suffice as a single predictor of customers’ future loyalty behaviour. • Use of a multiple indicator instead of a single predictor model performs better in predicting customer recommendations and retention. • Thus far, however, there have been no peer-reviewed, scientific investigations examining the relationship between recommend intention and customer behaviours (outside of customer referral/complaining behavior). Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet," Managing Service Quality (17:4), 2007, pp. 361-384 Slide 24
  • 25. Testing the Net Promoter Score® claims • FINDING: “The assertion that recommend intention alone will suffice as a predictor of customers‟ future loyalty behavior (Reichheld NPS), however, is not supported. We reach this conclusion based upon three primary findings. – First, bivariate correlations of all the attitudinal variables and customer behaviours investigated tended to be modest. – Second, when examining the three primary behaviours associated with customer loyalty (retention, share of wallet, and recommendations) recommend intention was generally not the best predictor for each of these variables. – Third, multivariate models universally outperformed models that use only recommend intention Source: Keiningham, T., Cooil, B., Aksoy, L., Andreassen, T. and Weiner, J. "The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet," Managing Service Quality (17:4), 2007, pp. 361-384 Slide 25
  • 26. Secure Customer Index as Net measure • The Secure Customer Index® probes three attributes1: – the “secure” customers were very satisfied, – had a likelihood to definitely continue using the service, – and had a likelihood of definitely recommending the service to others • Customers grouped into subgroups or loyalty segments Secure Favourable Vulnerable At Risk • Direct linkage to financial & market performance was calculated 1Brandt, D. (1996). Customer Satisfaction Indexing, Conference Paper presented at American Marketing Association, USA Slide 26
  • 27. Secure Customer Index (SCI) as net measure • Today the new improved SCI® is Burke Incorporated’s proprietary modelling approach • Five dimensions to assist validity and predictions of future share of wallet: Likelihood Likelihood Earned Overall Preferred to to Loyalty Satisfaction Company Recommend Repurchase • Burke has studied data which directly links and also projects a correlation between customer satisfaction, loyalty, and value to financial performance • Through projection and direct linkage, they can calculate which part of the marketing mix will bring the largest ROI Slide 27
  • 28. Customer Experience – A “deep ecological paradigm” shift (Fritjof Capra – The Web of Life, 1996) Slide 28
  • 29. Key Drivers of Loyalty Slide 29
  • 30. Outcomes of Improved Customer Experience Outcomes of Customer Experience Customer-Related Efficiency-Related Employee-Related Overall Performance- Outcomes Outcomes Outcomes Related Outcomes Behavioral Intentions Financial Customer Repurchase Price Perceptions & Performance Commitment Intentions Willingness to pay Nonfinancial Customer Performance Behaviours Behavioral Intentions are determined by Customer how the drivers of Customer Satisfaction Customer Loyalty & Word-of-Mouth & are managed <by implication measured> Repurchase Behaviour Complaining Behaviour Defection – this is the essence of Customer Experience Management Source: Luo, X & Homburg, C. April 2007 Neglected Outcomes of Customer Satisfaction. Journal of Marketing, Vol 71, Apr 2007 (0 133- 149) Slide 30
  • 31. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Dangers of Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 31
  • 32. Research Question • The popularity of the Net Promoter Score has highlighted the use of net measures in customer experience measurement • Considering the preceding literature review and discussion regarding different net measures, it is obvious that no single measure can be used successfully in measuring the complex constructs of customer experience, customer satisfaction and customer loyalty • This presentation will explore a quantitative model that integrates the “best-of-both-worlds” through a combined metrics of net measures and a multi-attribute measure of customer experience Slide 32
  • 33. Research Objectives The purpose of this study is to investigate the following three objectives: • Explore the use and application of Net Measures in the measurement of Customer Experience • Compare Net measures in terms of reliability, validity, predictive ability and practical application • Position Net Measures within the body of knowledge of multi-attribute Customer Experience Measurement theory and practise Slide 33
  • 34. Research Design & Data Collection • Meta-analysis on data collected over a time frame of more than 5 years, covering more than 1.5 million customer interviews across South Africa • Survey results have been consolidated from enterprise wide proprietary customer satisfaction surveys across a range of clients • For the purpose of this presentation (and reliability) the data is limited to results from surveys in the financial services industry in Southern Africa • Respondent selection for each of the surveys under consideration was quota-based from client contact lists on proportional stratified sample designs • At the time of the interview, the respondent was a current customer of the financial service provider being evaluated, and filter-controlled for having a recent interaction at a specific channel (enterprise-wide metrics across channels across segments) Slide 34
  • 35. Research Design & Data Collection • Survey data was collected via telephonic, web-based and face-to- face interviews • To ensure minimal non-sampling error, all interviews were subject to strict quality assurance processes, and advanced technology was used to capture data • No ethical issues are relevant to the study since most of the findings will be reported at meta-data levels without identifying any specific sponsoring company (to protect confidentiality and proprietary measures) • A strict ESOMAR code-of-conduct was followed in all data collection. The respondents were made aware of the institutions sponsoring the survey and for what purposes the information would be used Slide 35
  • 36. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Dangers of Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 36
  • 37. Research Methodology & Instruments • Prof Adré Schreuder developed a conceptual cause- and-effect model illustrated as an integrated customer experience measurement • Developed through years of academic research combined with extensive experience regarding Customer Satisfaction measurement across multiple industries • Basis for measurement is a structural model of customer satisfaction that incorporates the important constructs of satisfaction that will identify underlying service or product deficiencies (or strengths) and a proprietary algorithm for integrating net measures into this multi-attribute model Slide 37
  • 38. The CONSULTA Integrated Customer Experience Measurement Model FAILURE DELIGHT FAILURE DELIGHT FAILURE DELIGHT Slide 38
  • 39. The Conceptual Model Flow Copyright © Consulta Research - 2010 Slide 39
  • 40. Principle Calculation of Modeled Scores DELIGHT FAILURE Slide 40
  • 41. Instrument Development Process Slide 41
  • 42. Model Development Process Slide 42
  • 43. Use an Enterprise-wide Model – A Retail Banking example Slide 43
  • 44. Present CE Metrics in Dashboards Slide 44 Slide 44
  • 45. Research Methodology & Instruments The integrated customer experience measurement, although resulting in a final index score, acknowledges the fact that a single value for an index might hide more that it reveals It is important to be able to delve deeper into the results to enable the receiver to delve deeper than satisfaction For this reason the customer experience index score is not reported in isolation as a single number, but merely as the net result of multiple items, each of which contains detail results and offers valuable strategic information into the management of customer delight, loyalty, propensity to shift, service recovery, corrective improvement measures and consequence management Slide 45
  • 46. Research Methodology & Instruments • Research Instruments: – Same basic layout including sections corresponding to the components contained in the conceptual model for customer satisfaction measurement – First section measures specific channel’s value proposition with a range of custom designed service attributes - incorporates both customer perception and customer expectation by using confirmation-disconfirmation scale Much worse than expected Much better than expected 0 1 2 3 4 5 6 7 8 9 10 – Specific questions on product quality, service quality, relationship quality & pricing as contributing factors/components of customer satisfaction Slide 46
  • 47. Meta-data and Analysis • For each of the surveys the statistical analysis (using the statistical software package STATISTICA) included: – reliability and factor analysis; – structural equation modelling; – multiple regression analysis • The result, for each of the surveys, was a unique structural (cause- and-effect) model of customer satisfaction that considers all the important drivers of satisfaction • Final data set used for meta-analysis contained each of the components defined on next slide • Included 704 separate customer satisfaction studies forming part of the enterprise wide measurement of customer experience, for each of the financial institutions - each with a sample of at least 100 respondents and more Slide 47
  • 48. Meta-data and Analysis Metric Description Weighted service A weighted average of the (unique channel) service attribute average score attributes measured in terms of customer expectation Service problems % Proportion of respondents who indicated that they experienced a service problem within a certain time period. This is different from the proportion of respondents complaining (formally or informally) as measured in ACSI Problem recovery % Proportion of respondents who indicated that their service problem was recovered to their satisfaction Overall delight % Proportion of respondents who gave a 9 or 10 rating out of 10 for overall satisfaction. This is much more strict than the typical Top 2 Box metric calculated on a 5 point verbal scale or the „equivalent‟ “top four” boxes on the ten-point ACSI scale Slide 48
  • 49. Meta-data and Analysis Metric Description Overall failure % Proportion of respondents who gave a 0 or 1 rating out of 10 for overall satisfaction Average score (overall A simple average of overall satisfaction rated on a scale satisfaction) from 0 to 10 Customer satisfaction Index score (out of 100) is a function of the following key index score elements:  Underlying structural model  Basic calculation principle of being “rewarded” for positive ratings and being “penalised” for negative ratings – corresponding to the concept of a net measure Net Promoter Score Calculated according to the original definition of Reichheld (2003) the Net Promoter Score equals the % of promoters minus the % of detractors Slide 49
  • 50. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Dangers of Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 50
  • 51. The Dangers of Reporting Net Measures in Isolation • Danger/weakness in reporting any net measure (in isolation): two measurements having exactly the same value for the net measure can in fact have a range of different values assigned to the components of the net measure Slide 51
  • 52. The Dangers of Reporting Net Measures in Isolation • Recommendation not only applicable to net measures, but to other “simple” statistical measures (e.g. the sample mean) as well • A variety of different respondent values can also yield the same result for the specific statistical measure and typical “distribution” detail and/or graphs provide more insight into the results Slide 52
  • 53. Satisfaction Measures as Predictors of the NPS • As is to be expected, service problems and failure ratings show a negative correlation with customer satisfaction and NPS, while delight ratings show a positive correlation. Service problem recovery shows a very low, but positive, correlation with the NPS – NOTE poor R2 Slide 53 Sample Base: 1.5million respondents
  • 54. Satisfaction Measures as Predictors of the NPS Slide 54
  • 55. Satisfaction Measures as Predictors of the NPS • Individually, as independent variables in modelling the Customer Loyalty, the graphs and correlation coefficients clearly show that the integrated index score with an R2 of 0.73 seems to be the best predictor of the Net Promoter Score Slide 55 Sample Base: 1.5million respondents
  • 56. Satisfaction Measures as Predictor However, we do not recommend either the NPS or customer satisfaction index score in isolation as “the best and sufficient measurement to evaluate business performance”, but agree with Schneider et al. that “using a variety of measures rather than simply one measure would better capture the complexity underlying customer satisfaction and customer behaviours” Schneider, D.; Berent, M.; Thomas, R. & Krosnick, J. (2008). Measuring Customer Satisfaction and Loyalty: Improving the Net-Promoter Slide Score. Poster presented at the Annual Meeting of the American Association for Public Opinion Research, New Orleans, Louisiana 56
  • 57. Integrated Satisfaction Measure as Predictor The net measure(s) in itself can provide a top line measurement to track performance or even be effectively used as a “top-of-house” executive indicator Analysing the detail of all the different metrics constituting the customer satisfaction index score and NPS will assist greatly in the need for root cause analyses and strategic/tactical direction The quantitative data analysis of these measures is further enriched by qualitative questions similar to the “whys” asked by GE, including verbatim descriptions of service problems that were experienced, suggestions on improving service delivery, etc. Slide 57
  • 58. Normality of Customer Experience Modelled Score • Due to more complex nature of its calculation, efforts to examine statistical properties of net measures using a mathematical approach can be tedious and difficult • Computer-intensive simulation methods such as the bootstrap provide a solution • The bootstrap method was applied to replicate 1 000 bootstrap samples for each of four different studies – each bootstrap sample consisted of 380 respondents chosen randomly (with replacement) from the survey data • This provided 1 000 simulated index scores, which can be plotted as histograms and normal probability plots The accuracy of the simulations increase as the number of bootstrap replications increase; 500 or more simulations are sufficient to reduce variability and provide accurate results Slide 58
  • 59. Normality of Customer Experience Modelled Score Variable: VoC1, Distribution: Normal Normal Probability Plot of VoC1 (4 VoCs for normality graphs 4v*1000c) Chi-Square test = 8.67399, df = 9 (adjusted) , p = 0.46790 4 20 3 18 2 16 Expected Normal Value 14 1 Relative Frequency (%) 12 0 10 -1 8 -2 6 -3 4 2 -4 0 -5 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 50 52 54 56 58 60 62 64 66 68 Category (upper limits) VoC1: SW-W = 0.998084051, p = 0.3196 Observed Value Variable: VoC2, Distribution: Normal Normal Probability Plot of VoC2 (4 VoCs for normality graphs 4v*1000c) Chi-Square test = 5.06307, df = 7 (adjusted) , p = 0.65227 4 25 3 20 2 Expected Normal Value Relative Frequency (%) 1 15 0 10 -1 -2 5 -3 0 -4 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 40 42 44 46 48 50 52 54 Category (upper limits) VoC2: SW-W = 0.998708772, p = 0.6945 Observed Value Variable: VoC3, Distribution: Normal Normal Probability Plot of VoC3 (4 VoCs for normality graphs 4v*1000c) Chi-Square test = 8.15932, df = 7 (adjusted) , p = 0.31876 4 25 Slide 59 3
  • 60. Expect Relative 10 -1 -2 5 -3 Normality of Customer Experience Modelled Score 0 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 -4 40 42 44 46 48 50 52 54 Category (upper limits) VoC2: SW-W = 0.998708772, p = 0.6945 Observed Value Variable: VoC3, Distribution: Normal Normal Probability Plot of VoC3 (4 VoCs for normality graphs 4v*1000c) Chi-Square test = 8.15932, df = 7 (adjusted) , p = 0.31876 4 25 3 20 2 Expected Normal Value Relative Frequency (%) 1 15 0 10 -1 -2 5 -3 0 -4 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 28 30 32 34 36 38 40 42 44 Category (upper limits) VoC3: SW-W = 0.998033823, p = 0.2971 Observed Value Variable: VoC4, Distribution: Normal Normal Probability Plot of VoC4 (4 VoCs for normality graphs 4v*1000c) Chi-Square test = 6.36535, df = 7 (adjusted) , p = 0.49779 4 25 3 20 2 Expected Normal Value Relative Frequency (%) 1 15 0 10 -1 -2 5 -3 0 -4 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 32 34 36 38 40 42 44 46 Category (upper limits) VoC4: SW-W = 0.998200047, p = 0.3767 Observed Value Slide 60
  • 61. Normality of Customer Experience Modelled Score • For all four studies, both the chi-square test and Shapiro-Wilk test did NOT reject normality of the customer satisfaction index score, which holds the benefit of statistical inference of the index score (e.g. calculating confidence intervals and performing hypothesis testing) • Although these results are based on only four studies, representing a small portion of the wide range of underlying models used to describe the results of the various studies, we believe that with additional research we will be able to establish similar results for the whole range of studies under consideration, and consequently establish normality for the customer satisfaction index score in general Slide 61
  • 62. Index • Background & Rationale for Research • Previous Research and Literature Review • Research Question & Objectives • Research Methodology & Data Analysis • Research Results & Discussion – Reporting Net Measures in Isolation – Satisfaction Measures as Predictors of NPS – Normality of Customer Experience Modelled Score • Conclusion Slide 62
  • 63. Conclusion • Without denying the fact that net measures has a role to play, the use of net measures as standalone questions has been shown to have some disadvantages • Reporting net measures in context, supported by the multiple items it contains, provides the opportunity to analyse the detail of all the different metrics constituting the net measure • This assist in the need for root cause analyses and strategic/tactical direction, while the net measure in itself can provide a top line measurement to track performance or even be effectively used as a “top-of-house” executive indicator • The quantitative data analysis of these measures can further be enriched by qualitative questions, including verbatim descriptions of service problems that were experienced, suggestions on improving service delivery, etc. Slide 63
  • 64. Conclusion • Using longitudinal meta-data analysis of more than 1.5 million customer satisfaction measurement interviews, we have presented reliable correlations between the Net Promoter Score and an Integrated Customer Satisfaction Index score, as well as establishing statistical properties of these measures • The Customer Satisfaction Index score can be classified as a combined multi-attribute and net measure approach, since it incorporates the net effect of “failure” and “delight” ratings, as well as service problems and the recovery thereof Slide 64
  • 65. Conclusion Understanding that customers, as human beings, are complex by nature and accepting that the measurement of customer satisfaction involves the measurement of a complex construct, the use of an integrated measure of multiple-item & net measures has the advantage of providing insight into underlying drivers of customer satisfaction, while also offering a simple “top-of-house” dashboard metric that is simple to communicate. Slide 65