Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What MBA Students Need to Know about CX, Data Science and Surveys

2,164 views

Published on

I gave a talk to a group of executive MBA students at UW on the topic of customer experience.

Published in: Data & Analytics
  • Hello! Get Your Professional Job-Winning Resume Here - Check our website! https://vk.cc/818RFv
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

What MBA Students Need to Know about CX, Data Science and Surveys

  1. 1. Customer Experience, Big Data Analytics and Surveys
  2. 2. Bob E. Hayes, PhD Chief Research Officer Email: bob@appuri.com Web: www.appuri.com Twitter: @bobehayes • Author of three books on customer experience management and analytics • PhD in industrial-organizational psychology • #1 blogger overall on CustomerThink (http://customerthink.com/author/bobehayes/) • #1 blogger on the topic of customer analytics (http://customerthink.com/top-authors-category/) • Top expert in Big Data and Data Science • https://www.maptive.com/the-top-100-big-data- experts/ • http://www.kdnuggets.com/2015/02/top-big-data- influencers-brands.html
  3. 3. Appuri Help businesses improve retention, advocacy and growth Chief Research Officer Directing research on best practices in customer analytics, data science and measurement Business Over Broadway Solve problems through the use of the scientific method Owner Using data and analytics to help make decisions that are based on fact, not hyperbole What I do
  4. 4.  CX in a Big Data World  Optimal Customer Survey  Analytics of Survey Data  Two-Question Survey Contents
  5. 5. CX in a Big Data World
  6. 6. • A phenomenon about the quantification of everything • Different sides of Big Data: 1. Processing of Three Vs (volume, velocity, variety) 2. Insights (data science, veracity) 3. Analytics (types, data source, machine learning) 4. Data Integration (the sum of your data is greater than some of your data) 5. Communication (visualization, storytelling) 6. Security/Privacy/Ethics (data use policy) Big Data Image from Domo (2016)
  7. 7. • Interest in Big Data topics is growing dramatically • Relatively speaking, interest in customer experience shows slight growth • Good opportunity to incorporate big data principles (data science, machine learning) into CX programs Interest in Customer Experience and Big Data
  8. 8. • You have a lot of data about your customers. • Don’t rely on just surveys to understand and predict customer behaviors Your Big Data Data Format Structured Unstructured DataSource Internal Human-Generated • Survey ratings • Aptitudetesting Machine-Generated • Web metrics from Web logs • Product purchase from sales Records • Process control measures Human-Generated • Emails, letters, text messages • Audiotranscripts • Customer comments • Voicemails • Corporate video/ communications • Pictures, illustrations • Employeereviews External Human-Generated • Number of Retweets, Facebook likes, Google Plus+1s • Ratings on Yelp • Patient ratings Machine-Generated • GPS for tweets • Time of tweet/ updates/ postings Human-Generated • Content of social media updates • Comments in onlineforums • Comments on Yelp • Video reviews • Pinterest images • Surveillance video
  9. 9. • The goal is to know everything about each customer • Your analytics will result in better predictive models for all customers • Lead to true CX personalization Integrate Your Data
  10. 10. Value from Analytics: MIT / IBM 2010 Study Top-performing organizations use analytics five times more than lower performers http://sloanreview.mit.edu/the-magazine/2011- winter/52205/big-data-analytics-and-the-path-from- insights-to-value/
  11. 11. Data Integration is Key to Extracting Value - Operational 96% 72% 51% 50% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Percent of VOC executives who are satisfied with program Customer loyalty percentile rank (within industry) PercentofVOCExecutives/ CustomerLoyaltyPercentileRank Ops Linkage Analysis No Ops Linkage Analysis
  12. 12. A way of extracting insights from data using the powers of computer science and statistics applied to data from a specific field of study Goal: empirically-based insights that augment and enhance human decisions and algorithms What is Data Science?
  13. 13. Skills of Data Science Area Skills* Business 1.Product design and development 2.Project management 3.Business development 4.Budgeting 5.Governance & Compliance (e.g., security) Technology 6.Managing unstructured data (e.g., noSQL) 7.Managing structured data (e.g., SQL, JSON, XML) 8.Natural Language Processing (NLP) and text mining 9.Machine Learning (e.g., decision trees, neural nets, Support Vector Machine, clustering) 10.Big and Distributed Data (e.g., Hadoop, Map/Reduce, Spark) Math & Modeling 11.Optimization (e.g., linear, integer, convex, global) 12.Math (e.g., linear algebra, real analysis, calculus) 13.Graphical Models (e.g., social networks) 14.Algorithms (e.g., computational complexity, Computer Science theory) and Simulations (e.g., discrete, agent-based, continuous) 15.Bayesian Statistics (e.g., Markov Chain Monte Carlo) Programming 16.Systems Administration (e.g., UNIX) and Design 17.Database Administration (MySQL, NOSQL) 18.Cloud Management 19.Back-End Programming (e.g., JAVA/Rails/Objective C) 20.Front-End Programming (e.g., JavaScript, HTML, CSS) Statistics 21.Data Management (e.g., recoding, de-duplicating, Integrating disparate data sources, Web scraping) 22.Data Mining (e.g. R, Python, SPSS, SAS) and Visualization (e.g., graphics, mapping, web-based data visualization) tools 23.Statistics and statistical modeling (e.g., general linear model, ANOVA, MANOVA, Spatio-temporal, Geographical Information System (GIS)) 24.Science/Scientific Method (e.g., experimental design, research design) 25.Communication (e.g., sharing results, writing/publishing, presentations, blogging) * List of skills adapted from Analyzing the Analyzers by Harlan D. Harris, Sean Patrick Murphy and Marck Vaisman
  14. 14. The Skills of Data Science
  15. 15. Not all Data Scientists are Created Equal • Different types of data scientists possess different skills • Biz Management – strong in business skills • Developer – strong in technology/programming skills • Researcher – strong in math/ statistics skills • Creatives – average in all skills
  16. 16. 1. Formulate Questions 2. Generate hypothesis/ hunch 3. Gather / Generate data 4. Analyze data / Test hypothesis 5. Take action / Communicate results • Start with a problem statement. • What are your hunches / hypotheses? • Be sure your hypotheses are testable. • You can use experimental or observational approach to analyzing data. • Integrate your data silos to ask bigger questions; connect the dots and get a 360 degree view of your customers. • Employ Predictive analytics / Inferential statistics to test hypotheses • Employ machine learning to quickly surface insights • Implement your findings • Use Prescriptive analytics to guide course of action From Questions to Actions: The Scientific Method
  17. 17. A Team Sport: Data Science Skills and the Scientific Method
  18. 18. Customer Analytics Maturity Matrix Fundamental Awareness Intermediate Advanced Data used to describe current state of customer health Data used to understand why things happened Data used to manage specific customers Data used to identify drivers of customer behaviors Maturity Stage Process World Class Beginner Data used to improve systemic problems that improve the health of all customers • Deploy algorithms developed by your data scientists • Employ sophisticated analytics to uncover customer insights (exploratory) • Analyze data using machine learning to identify drivers of churn • Integrate customer insights into existing sales / marketing automation systems (e.g., risk scoring Accounts/Contacts) • Create dashboards to understand what happened in the previous time periods • Integrate data silos to create a Unified Customer Profile - providing holistic, 360 degree, view of customers PrescriptivePredictiveDiagnosticDescriptive StrategicUseof CustomerData ManualProcessesAutomatedProcesses TacticalUseof CustomerData
  19. 19. Customer Analytics Maturity Matrix * Preliminary results from recent study on customer analytics best practices. For a free assessment of your CX or Customer Success program, take the study survey by clicking here: http://bit.ly/cabpa.
  20. 20. Optimal Customer Survey
  21. 21. Asking the right questions leads to deeper customer insights: 1. How loyal are the customers to the company? Will customers be engaging in different types of loyalty behaviors (e.g., recommend, buy different products/services, expand usage, renew service contracts)? 2. How satisfied are the customers with the customer experience? Are customers satisfied with different touch points (e.g., product, ease, support, communication)? 3. How does the company rank against the competition? Do customers think the company is the best/worst/typical in the industry? 4. What is the general sentiment of your customers? 5. Where would CX improvement efforts have the biggest ROI? If you purchase the company, what do you need to fix first? 6. Customer Relationship Diagnostic (CRD) Customer Relationship Surveys Help you Answer Important Questions
  22. 22. Customer Loyalty
  23. 23. 1. Retention – will customers stay/churn? 2. Advocacy – will customers recommend? 3. Purchasing – will customer expand relationship Customer Loyalty Drives Business Growth/Value Company growth/value is impacted by three types of customer behavior:
  24. 24. Customer Loyalty Questions Type Definition Loyalty Questions Retention Loyalty The degree to which customers will remain as a customer/not leave to competitor 1. Likelihood to switch to another company* 2. Likelihood to purchase from competitor* 3. Likelihood to renew service contract Advocacy Loyalty The degree to which customers feel positively toward/will advocate your product / service / brand 4. Overall satisfaction 5. Likelihood to recommend (NPS) 6. Likelihood to purchase same product/service Purchasing Loyalty The degree to which customers will increase their purchasing behavior 7. Likelihood to purchase different/additional products/services 8. Likelihood to expand use of products across company 0 1051 2 3 4 6 7 8 9 Not at all Likely Extremely Likely
  25. 25. Consider Objective Loyalty Metrics Measurement Approach Objective Subjective (Survey Questions) LoyaltyTypes Emotional ADVOCACY • Number/Percent of new customers • Social media engagement - Likes/Shares ADVOCACY Intentions • Overall satisfaction • Recommend • Buy same product • Level of trust • Willing to forgive • Willing to consider RETENTION Intentions • Renew service contract • Stay or Leave PURCHASING Intentions • Buy different/additional products • Likelihood to expand usage Behavioral RETENTION • Churn rates • Service contract renewal rates PURCHASING • Usage – Frequency of use, Page views • Sales Records - Number of products purchased
  26. 26. Customer Experience
  27. 27. 1. Sum of all experiences a customer has with a supplier of goods or services, over the duration of their relationship with that supplier 2. The quality of the customer experience is measured through satisfaction ratings 3. Understand drivers of customer loyalty Better customer experience leads to higher levels of customer loyalty Satisfaction with the Customer Experience
  28. 28. General vs. Specific CX Questions
  29. 29. General CX Questions Predict Loyalty Well; Specific Questions Add Little
  30. 30.  Overall, how satisfied are you with each area? 1. Ease of doing business 2. Sales / Account Management 3. Product Quality 4. Service Quality 5. Technical Support 6. Communications from the Company 7. Future Product/Company Direction General Customer Experience Questions
  31. 31. CX has greater impact on advocacy loyalty .00 .10 .20 .30 .40 .50 .60 .70 .80 .90 Ease of doing business Overall Product Quality Responsiveness to Service Needs Responsiveness to Technical Problems Ability to Resolve Technical Problems Communications from the Company Future Product/Company Direction ImpactonLoyaltyMetric (correlationbetweenbusinessattributes andloyaltymetric) Advocacy Loyalty Purchasing Loyalty Retention Loyalty 1 Importance measured by correlation between business attribute and customer loyalty metric. Ranking conducted within a specific loyalty metric.
  32. 32. Benchmarking
  33. 33.  Customer experience questions may not be enough to improve business growth  You need to understand your relative performance  HBR study (2011)1: Top-ranked companies receive greater share of wallet compared to bottom- ranked companies  If customers think you’re the best, they will deepen their buying relationship with you Benchmarking: Competitive Analytics 1 Keiningham, Timothy L., Lerzan Aksoy, Alexander Buoye, and Bruce Cooil (2011), “Customer Loyalty Isn’t Enough. Grow Your Share of Wallet.” Harvard Business Review. vol. 89 (October).
  34. 34. Loyalty Benchmarks (B2B) 0 1 2 3 4 5 6 7 8 9 10 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Mean PercentofRespondents Very Loyal (ratings of 9 or 10) Loyal (ratings of 6-8) Disloyal (ratings of 0-5) Mean Overall Satisfaction Recommend Continue Purchasing / Using Select Vendor Again Means not calculated for overall sat for Comp D. Comp D used a 1 to 6 satisfaction scale; 1-3 = Disloyal; 4-5 = Loyal; 6 = Very loyal. Comp A provides consulting services, analytics solutions and customized products for financial companies. Comp B provides solutions that help healthcare providers automate key business processes. Comp C helps businesses monitor and optimize Storage Area Networks (SANs). Comp D provides security and data protection solutions. Comp E specializes in developing hardware systems and enterprise software products. Comp F provides solutions for precision electrical measurement and test of advanced semiconductor devices.
  35. 35.  What best describes our performance compared to the competitors you use? Bootstrap Benchmarking: Relative Performance
  36. 36. CRD: The Survey Customer Loyalty and Customer Experience Indices / Measures1 Addresses Business Growth Survey Questions Scores Customer Loyalty (RAPID) Retention Loyalty Index (RLI) Advocacy Loyalty Index (ALI) Purchasing Loyalty Index (PLI) RLI: Will your customers remain with / not leave you? ALI: Will your customers promote you? PLI: Will your customers invest in additional product / service offerings? RLI: Renew service contract, Use competitor2 ALI: Overall Satisfaction, Recommend, Continue purchasing / using PLI: Purchase additional services, Expand usage Scores for each index can range from 0 (low loyalty) to 10 (high loyalty) General Customer Experience (GENCX) Ease of doing business, Account Mgmt, Product Quality, Customer Service, Tech Support Communications from Company, Future Product/Company Direction Are your customers receiving a great customer experience? Customers provide satisfaction rating for each of the 7 business areas. Scores can range from 0 (high dissatisfaction) to 10 (high satisfaction) Relative Performance Assessment (RPACX) Are you ahead of the competition? How does your company perform relative to the competition? Scores can range from 0 (low ranking) to 100 (high ranking) Customer Sentiment (CSI) Do your customers have a generally positive or negative opinion of your brand? What one word best describes this company?3 Scores can range from 0 (negative sentiment) to 100 (positive sentiment) 1 Indices are the average rating. RAPID ratings are calculated by averaging over the questions in a specific index. Each index has been shown to have a high degree of reliability.2 Reverse coded so higher ratings reflect high retention loyalty. 3 Scaled using sentiment lexicon.
  37. 37. Analytics of Survey Data
  38. 38. 1. Descriptive: What happened? Mean, Standard Deviation, Frequencies 2. Predictive: What will happen? Correlation, Regression, Clustering 3. Prescriptive: What should I do? Combination of Descriptive and Predictive and Business Rules (logic) Three Types of Analytics 1% 3% 25% 54% 17% 0% 10% 20% 30% 40% 50% 60% Terrible Poor Fair Good Excellent PercentofRespondents 46% 47% 42% 34% 34% 33% 28% 44% 37% 41% 46% 46% 47% 48% 10% 17% 16% 20% 20% 20% 24% 0% 20% 40% 60% 80% 100% PercentofCustomers Very Satisfied Satisfied Dissatisfied GENCX Index 75
  39. 39. Executive Dashboard: Customer Loyalty 57% 34% 43% 25% 28% 51% 31% 53% 46% 38% 44% 33% 12% 13% 11% 38% 29% 15% 0% 20% 40% 60% 80% 100% Same Sat Rec Add Expand Renew PercentofCustomers Very Loyal Loyal At Risk Reten- tion (Renew) Advocacy (Buy same Sat, Rec) Purchasing (Buy add, expand) Advocacy 79 Retention 79 Purchasing 66 RAPID Index 75 • Majority of customers report high levels of loyalty across different loyalty types / lowest for Purchasing Loyalty • Loyalty improves over prior years 76 79 79 74 75 79 55 60 66 25 50 75 100 2011 2012 2013 CustomerLoyalty Retention Advocacy Purchasing
  40. 40. Executive Dashboard: Customer Experience 46% 47% 42% 34% 34% 33% 28% 44% 37% 41% 46% 46% 47% 48% 10% 17% 16% 20% 20% 20% 24% 0% 20% 40% 60% 80% 100% PercentofCustomers Very Satisfied Satisfied Dissatisfied GENCX Index 75 Account Manage-ment 65 Ease of Doing Business 69 Communi- cation 69 Direction and Future 78 Customer Service 77 Technical Support 77 Product Quality 79 Majority of customers report they are satisfied with Company ABC
  41. 41. Analytics Workflow Data Predictive what will happen? Descriptive what happened? Prescriptive what should I do? Decisions Actions
  42. 42. Need to know two things about each CX touch point: 1. Level of customer satisfaction or performance (descriptive) 2. Importance to / Impact on customer loyalty (predictive) Descriptive and Predictive Analytics for your Customer Survey CX Touch Point Performance1 (Average CS Rating) Impact2 on Advocacy Loyalty Ease of doing business 6.90 .72 Overall Product Quality 6.65 .83 Responsiveness to Service Needs 7.08 .62 Responsiveness to Technical Problems 7.19 .62 Ability to Resolve Technical Problems 7.03 .59 Communications from the Company 6.68 .64 Future Product/Company Direction 5.69 .64 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 0 (Extremely Dissatisfied) to 10 (Extremely Satisfied); 2 Impact is the correlation between specific CX Touch Point and Advocacy Loyalty Index.
  43. 43. 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 5.25 5.75 6.25 6.75 7.25 7.75 ImpactonAdvocacyLoyalty (correlationbetweenbusinessattributes andAdvocacyLoyaltyIndex) Performance on Business Attribute (Customer Rating) Loyalty Driver Matrix: Prescriptive Analytics Advocacy Loyalty Index is the average of the following four questions (Overall Satisfaction, Recommend, Select vendor again, Continue using). Where should we invest? 1. For each CX touch point, plot: performance by impact on loyalty 2. Apply business rule to plot (prescribes course of action)
  44. 44. Prescriptive Analytics – Loyalty Driver Matrix Examine each CX touch point’s performance and impact on loyalty simultaneously. 1. Key Drivers – Invest in areas to increase Customer Loyalty. 2. Hidden Drivers – Use features in marketing to grow customer base. 3. Visible Drivers – Consider features in marketing to grow customer base. 4. Weak Drivers – Monitor as lowest priority for investment.
  45. 45. Making Improvements: Predicting Advocacy Loyalty Advocacy Loyalty Index is the average of the following four questions (Overall Satisfaction, Recommend, Select vendor again, Continue using). 0.50 0.55 0.60 0.65 0.70 0.75 0.80 0.85 5.25 5.75 6.25 6.75 7.25 7.75 ImpactonAdvocacyLoyalty (correlationbetweenbusinessattributes andAdvocacyLoyaltyIndex) Performance on Business Attribute (Customer Rating) To improve advocacy loyalty, you may consider focusing on following areas: 1. Overall Product Quality
  46. 46. CX Touch Point Performance1 (Average Rating) Impact2 on Advocacy Loyalty Ease of doing business 6.90 .72 Overall Product Quality 6.65 .83 Responsiveness to Service Needs 7.08 .62 Responsiveness to Technical Problems 7.19 .62 Ability to Resolve Technical Problems 7.03 .59 Communications from the Company 6.68 .64 Future Product/Company Direction 5.69 .64 ImproveLeverage Consider 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 0 (Extremely Dissatisfied) to 10 (Extremely Satisfied); 2 Impact is the correlation between specific Business Attribute sand Advocacy Loyalty Index.  Improving Advocacy Loyalty Improving and Marketing ACME
  47. 47. Exercise
  48. 48. Example: Driver Analysis for Purchasing Loyalty CX Touch Point Performance1 (Average CS Rating) Impact2 on Purchasing Loyalty Ease of doing business 6.90 .29 Overall Product Quality 6.65 .44 Responsiveness to Service Needs 7.08 .25 Responsiveness to Technical Problems 7.19 .33 Ability to Resolve Technical Problems 7.03 .35 Communications from the Company 6.68 .40 Future Product/Company Direction 5.69 .39 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 0 (Extremely Dissatisfied) to 10 (Extremely Satisfied). 2 Impact is the correlation between specific CX Touch Point and Purchasing Loyalty.
  49. 49. Example: Driver Analysis for Retention Loyalty CX Touch Point Performance1 (Average CS Rating) Impact2 on Retention Loyalty Ease of doing business 6.90 .19 Overall Product Quality 6.65 .26 Responsiveness to Service Needs 7.08 .19 Responsiveness to Technical Problems 7.19 .27 Ability to Resolve Technical Problems 7.03 .27 Communications from the Company 6.68 .17 Future Product/Company Direction 5.69 .30 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 0 (Extremely Dissatisfied) to 10 (Extremely Satisfied). 2 Impact is the correlation between specific CX Touch Point and Retention Loyalty.
  50. 50. CX Touch Point Performance1 (Average Rating) Impact2 on Purchasing Loyalty Ease of doing business 6.90 .29 Overall Product Quality 6.65 .44 Responsiveness to Service Needs 7.08 .25 Responsiveness to Technical Problems 7.19 .33 Ability to Resolve Technical Problems 7.03 .35 Communications from the Company 6.68 .40 Future Product/Company Direction 5.69 .39 ImproveLeverage Consider 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 0 (Extremely Dissatisfied) to 10 (Extremely Satisfied); 2 Impact is the correlation between specific Business Attribute sand Purchasing Loyalty Index.  Improving Purchasing Loyalty Improving and Marketing ACME
  51. 51. Making Improvements: Predicting Purchasing Loyalty Purchasing Loyalty Index is the average of the following two questions (Purchase different or new, Expand usage). To improve purchasing loyalty, you may consider focusing on following areas: 1. Communications from the Company 2. Overall Product Quality 3. Future Product/ Company Direction 0.20 0.25 0.30 0.35 0.40 0.45 0.50 5.25 5.75 6.25 6.75 7.25 7.75 ImpactonPurchasingLoyalty (correlationbetweenbusinessattributes andPurchasingLoyaltyIndex) Performance on Business Attribute (Customer Rating)
  52. 52. 1 Performance of each attribute is the average rating for each attribute across all respondents. Possible scores range from 1 (Extremely Dissatisfied) to 10 (Extremely Satisfied); 2 Impact is the correlation between specific Business Attribute sand Customer Loyalty.  Improving Retention Loyalty CX Touch Point Performance1 (Average Rating) Impact2 on Retention Loyalty Ease of doing business 6.90 .19 Overall Product Quality 6.65 .26 Responsiveness to Service Needs 7.08 .19 Responsiveness to Technical Problems 7.19 .27 Ability to Resolve Technical Problems 7.03 .27 Communications from the Company 6.68 .17 Future Product/Company Direction 5.69 .30 ImproveLeverage Consider Improving and Marketing ACME
  53. 53. Making Improvements: Predicting Retention Loyalty To improve retention loyalty, you may consider focusing on following areas: 1. Overall Product Quality 2. Future Product/ Company Direction Retention Loyalty Index is the average of the following question (Stop using) 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 5.25 5.75 6.25 6.75 7.25 7.75 ImpactonRetentionLoyalty (correlationbetweenbusinessattributes andRetentionLoyaltyIndex) Performance on Business Attribute (Customer Rating)
  54. 54. Two-Question Survey
  55. 55. 1. What one word best describes this company/product/service? 2. If you were in charge of this company, what improvements, if any would you make?
  56. 56. 56 Approaches to Measuring Attitudes Structured Data generated to measure specific construct How satisfied are you with company? Customer-generated Unstructured Data are given to us. We take what we can get emails, social media, support calls, movie reviews, tweet content, transcripts of comments Algorithm-generated IntentSourceScore
  57. 57. What one word best describes company’s products/service? Word cloud based on 944 respondents who answered the question, “What one word best describes the company’s products / services?” Font size of words corresponds to the frequency of words used by customers. Larger words are used more frequently by customers than smaller words.
  58. 58. Judgment-Based • Two subject matter experts independently rate list of words from customer survey 0 (negative sentiment) to 10 (positive sentiment) • High agreement between experts (rater) Lexicon Mean SD 1 2 3 4 1. Rater 1 First Rating 6.49 2.32 2. Rater 2 First Rating 6.44 1.93 .87 3. Rater 1 Second Rating 6.35 2.39 .98 4. Rater 2 Second Rating 6.42 1.95 .99 .90 5. Average Sentiment 6.39 2.11 .96 .96 .98 .97 N = 251. All correlations are statistically significant at the p < .01 level. Average sentiment: Based on based on the average second sentiment ratings of each rater. Bold correlations represent inter-rater agreement. Italic correlations represent intra-rater agreement. Descriptive Statistics Correlations
  59. 59. Empirically-Based • Examine four corpora (a collection of written text) with accompanying ratings • Data from four review sites* OpenTable, IMDB, Goodreads, Amazon/TripAdvisor * See Christopher Potts: http://web.stanford.edu/~cgpotts/talks/potts-wordnetmods.pdf
  60. 60. Description of Four Lexicon Sources
  61. 61. Calculating Sentiment for Each Word • Re-scale values from 1 to 5 -> 0 to 10 • Calculate sentiment of each word (adjective) “excellent” sentiment value of 8.39 “good” sentiment value of 6.69
  62. 62. Distribution of Words’ Sentiment Values
  63. 63. Descriptive Statistics and Correlations of Sentiment Values of Words
  64. 64. Difference among Lexicons
  65. 65. Reliability of Customer Sentiment Index • B2B Technology Company Customer Survey – one word, ratings • Context is important Mean SD N 1 2 3 4 1. CSI - Expert 7.09 1.84 894 2. CSI - OpenTable 7.12 1.18 766 .77 3. CSI - IMDB 6.78 .86 786 .60 .78 4. CSI - Goodreads 6.30 1.23 757 .62 .74 .93 5. CSI - Amason/Tripadvisor 7.65 .97 623 .65 .83 .77 .68 Correlations among CSI scores Mean SD N CSI Expert CSI OT CSI IMDB CSI GR CSI A/TA Overall Satisfaction 7.60 1.99 1595 .57 .48 .33 .30 .43 Recommend 7.91 1.96 1585 .56 .49 .35 .31 .42 Purchase same / similar 8.22 2.26 1527 .34 .31 .24 .24 .21 Purchase additional / different 6.32 2.83 1508 .18 .16 .10 .09 .12 Expand use 6.80 2.59 1523 .22 .22 .18 .16 .18 Renew service contract 7.90 2.53 1159 .30 .25 .19 .18 .21 Ease of doing business 7.37 2.17 1204 .55 .49 .37 .35 .45 Account Management 6.98 2.32 1189 .42 .35 .26 .25 .36 Product Quality 7.93 1.89 1297 .50 .44 .27 .27 .42 Service / Repair 7.67 2.15 1092 .41 .37 .29 .30 .30 Technical Support 7.73 2.29 1253 .43 .37 .29 .29 .33 Communications from Company 7.37 2.13 1282 .51 .45 .34 .32 .39 Direction and future products/services 7.45 1.96 1165 .48 .41 .26 .26 .37 All correlations statistically significant at the p < .05 level. All measures are on a scale from 0 (low loyalty/satisfaction) to 10 (high loyalty/satisfaction). Based on respondents (N = 1619) of annual customer survey of a B2B technology company. Loyalty Satisfactionwiththe CustomerExperience Correlations of CSI scores with Loyalty/CX Metrics
  66. 66. Validity of Customer Sentiment Index Mean SD N CSI Expert CSI OT CSI IMDB CSI GR CSI A/TA Overall Satisfaction 7.60 1.99 1595 .57 .48 .33 .30 .43 Recommend 7.91 1.96 1585 .56 .49 .35 .31 .42 Purchase same / similar 8.22 2.26 1527 .34 .31 .24 .24 .21 Purchase additional / different 6.32 2.83 1508 .18 .16 .10 .09 .12 Expand use 6.80 2.59 1523 .22 .22 .18 .16 .18 Renew service contract 7.90 2.53 1159 .30 .25 .19 .18 .21 Ease of doing business 7.37 2.17 1204 .55 .49 .37 .35 .45 Account Management 6.98 2.32 1189 .42 .35 .26 .25 .36 Product Quality 7.93 1.89 1297 .50 .44 .27 .27 .42 Service / Repair 7.67 2.15 1092 .41 .37 .29 .30 .30 Technical Support 7.73 2.29 1253 .43 .37 .29 .29 .33 Communications from Company 7.37 2.13 1282 .51 .45 .34 .32 .39 Direction and future products/services 7.45 1.96 1165 .48 .41 .26 .26 .37 All correlations statistically significant at the p < .05 level. All measures are on a scale from 0 (low loyalty/satisfaction) to 10 (high loyalty/satisfaction). Based on respondents (N = 1619) of annual customer survey of a B2B technology company. Loyalty Satisfactionwiththe CustomerExperience Correlations of CSI scores with Loyalty/CX Metrics
  67. 67. Relationship between CSI and Recommend B2B Survey B2C Survey
  68. 68. What improvements would you make? 30% 16% 15% 8% 7% 6% 6% 5% 4% 4% 3% 3% 2% PercentofRespondents Data are based on 598 respondents who answered the following question: If you were in charge of company, what improvements, if any, would you make? Categories of improvement areas do not include nine categories because they were mentioned less than 2% of the times by the respondents. Frequency distribution based on 598 respondents who answered the question, “If you were in charge of the company, what improvements, if any, would you make?”
  69. 69. What improvements would you make? 7.24 6.89 6.85 6.81 6.80 6.80 6.80 6.79 6.77 6.38 5.98 5.88 5.36 CustomerSentiment Data are based on 598 respondents who answered the following question: If you were in charge of company, what improvements, if any, would you make? Categories of improvement areas do not include nine categories because they were mentioned less than 2% of the time by the respondents. Consider these touch points (arrowed) as a starting point to make improvements; customers who mention these improvement areas report significantly lower sentiment than customers who do not mention these areas.
  70. 70. Apply results across the company 1. Sales, Marketing & Service • use popular words in sales/marketing efforts to improve how collateral resonates with them 2. Product Management • use sentiment index in design thinking process to improve “testing” step 3. Operations • identify business areas/processes that need attention
  71. 71. Applications of Two-Question Survey • Mobile Surveys • Extract more information from single word Apply different lexicons (e.g., anxiety, strength) • Simplify feedback process Shorter surveys benefit customers Fewer dashboard metrics facilitate executive reports
  72. 72. Executive Dashboard: Customer Sentiment Index CS Index 70 What one word best describes this company? • 77% of customers have positive sentiment • 6% of customers have negative sentiment 4.9% 1.6% 16.4% 25.0% 52.0% Very Negative Sentiment - 0 through 2.5 Slightly Negative Sentiment - 2.6 through 4.5 Neutral Sentiment - 4.6 through 5.5 Slightly Positive Sentiment - 5.6 through 7.5 Very Positive Sentiment - 7.6 through 10 PercentofRespondents
  73. 73. Executive Dashboard: Customer Sentiment What one word best describes this company? What improvements, if any, would you make? CS Index 70
  74. 74. Further Reading To learn more about customer-centric measurement and analytics, RAPID Loyalty measurement, CX measurement and problems with the NPS, check out these books.

×