This paper's objective was to recognize the fallacies of Net Promoter Score and propose an alternative customer loyalty model using big data techniques. The proposed model assesses and predicts customer loyalty using attitudinal, behavioral and demographic data.

1. service800.com Confidentialservice800.com
Confidential documentConfidential document
University of Cambridge :Cambridge Service
Alliance
Mohamed Zaki,Dalia Kandeil, Andy
Neely,Janet R.McColl-Kennedy
The Fallacy of the Net
Promoter Score: Customer
Loyalty Predictive Model

2. service800.com Confidential
• S800 27 year CX expert in Customer Loyalty, Customer Relationship and Tracking Studies for
Technology Delivery organizations like
• Based in Minneapolis , London and Shanghai, collecting feedback from customers around the
world in 30 languages every day.
SERVICE 800

3. service800.com Confidential*Interview defined as at least the Overall Rating
• This paper provides a framework that holistically assesses and predicts customer loyalty using attitudinal and
behavioral data sources.
• The proposed predictive analytics model uses big data techniques to predict customer loyalty and identifies
customers who are no longer conducting business with the organization.
• The linguistic text mining approach determines the complaint status and emotions of each customer using
text-mining textual feedback that divides customers in to groups of complainers, neutral or satisfied.
• The model utilizes customers verbatim comments to understand WHY customers are churning.
Objective

4. service800.com Confidential
• A study claims that there are no real indication that average levels of attitudinal loyalty metrics significantly
correlate with the relative change in revenue within the industries. It also confirmed that the recommended
single intention measure alone is not sufficient to assess customer loyalty (Keiningham, 2007).
• The NPS is not sufficient as an approach to customer loyalty measurement and management because
recommendation alone are unable to drive business success (Grisaffe, 2007)
• There is no empirical evidence to support predictive ability of NPI on growth and financial performance
(Pollack and Alexandrov, 2013).
• Firms should not focus solely on customer feedback systems and customer recommendation intentions and
behavior metrics (Morgan and Rego,2006)
NPS Criticism

5. service800.com Confidential
• Three years worth of data points were collected (2012-2015), with a total of 1,044,512 transactional
records.
• B2B customer loyalty framework was developed with both the same and multiple customers, using multiple
touchpoints.
• Using RFM technique to transform customer transactional data into profitability scores, categorizing
customers based on purchasing behavior.
• K-means clustering technique was used to identify customers who had churned and where no longer doing
business with the company.
• A linguistic text-mining approach was used to categorize customer survey comments in to complainer,
neutral and satisfied.
• Lastly, the prediction model used neural network and Bayesian network algorithms to predict customer
loyalty.
Methodology

6. service800.com Confidential
Methodology

7. service800.com Confidential
• Customer purchasing patterns were observed over a predefined period. This was done by sorting customers
based recency values and then dividing the customer base into five groups. Each customer then receives a
rank, where the lowest recency values receive a score of 5, with the highest value receives a score of 1.
• This step is repeated for both frequency and monitory values, with the difference between the highest
values being assigned a score of 5.
• Finally each customer receives an overall RFM score, which is the combination of each RFM variable rank.
RFM Analysis

8. service800.com Confidential
• Using the K-means segmentation algorithm customers were divided into 11 groups based on their RFM
scores.
• Each K-means cluster has an average RFM score, which is the mean scores of all customers belonging to that
group.
• A total of 11 clusters was chosen to mimic the 11 point scale of the NPS, which would allow the
classification of customers based on NPS’s promoter, passive and detractor categories.
• The 11 customer groups were sorted based on their average RFM score and given a corresponding NPS
scale.
New NPS_RFM Categories

9. service800.com Confidential
• The study identified “active customers” as those whose time in days since their most recent transaction does
not exceed the average number of days of the difference between the two most recent consecutive
transactions across all customers.
• Each customer’s recency value is compared to the average number of days of the difference between the
two most recent consecutive transactions.
• If the value exceeds the average, the customer is labeled as a churner; otherwise, the customer is considered
loyal.
• Geographical location of the customers were used to pinpoint areas with higher loyalty and churn rates.
Active Customers

10. service800.com Confidential
• A text – mining model was implemented to analyze open ended questions to extract insightful information
about the customer experience.
• This sentence level analysis developed a 213 linguistic patterns across service processes (eg. field, parts, sales
etc.). Then each sentence was categorized as a complaint, a compliment or a suggestion.
• Using K-means segmentation algorithm a sentiment score was calculated for each comment.
• A total of 11 clusters were chosen to mimic the 11 point NPS scale, which enabled the classification of
customers NPS promoter, passive and detractor categories.
• Similar to the newly calculated RFM_NPS score, the 11 customer groups were then sorted based on their
average sentiment score.
• Satisfied customers were assigned to clusters 3 or 10.
• Neutral customers were assigned to clusters 5,7 and 11.
• Complainer customers were assigned under 1,2,4,6,8 and 9.
Text-mining Model

11. service800.com Confidential
• First the authors analyzed the loyalty status of surveyed customers with an unchanged value of NPS in the
three years. They found out that 70% of the customers were promoters, 25% passive and 5% as detractors.
They further looked into the customers who had unchanged loyalty status throughout the three years and
98% of them were unchanged promoters with a 2% passive loyalty.
• Second, a longitudinal analysis showed changes that occurred in the NPS over time, between two
consecutive years (2012–13 and 2013–14). The authors analyzed those who had different NPS scores from
one year to the next and split them into positive changes and negative changes.
• Third, based on behavioral data two types of customers were identified as customers with ongoing
Customer Service Agreement (CSA) customers; and those who deal in a non-contractual setting, referred to
as Product Support (PS) customers. They found that customers who had made a sales transaction
simultaneously made either a PS or a CSA.
• All CSA customers made a PS transaction except one customer.
Results- Descriptive Analytics

12. service800.com Confidential
• The customer transactions were broken in to two categories. Graph below explains the split between PS
parts and services.
Results- Descriptive Analytics

13. service800.com Confidential
• Having developed an RFM score categorization similar to that of the Net Promoter Score (NPS), it then
became possible to link the true underlying customer purchasing behavior with their self proclaimed referral
intention present in the survey’s NPS category.
• This process entailed comparing customers’ resulting new NPS_RFM with their actual NPS categories present
in the customer feedback survey data, to determine whether or not they matched.
• In total, the percentage of NPS misclassification in 2012 was 72% and 85% in 2013 and 82% in 2014. The
number of negative misclassifications exceeded the amount of positive ones.
• The results suggests that using a survey-based single-item metric is inadequate to determine customer
behavioral patterns.
Predictive Analytics

14. service800.com Confidential
• In another analysis the study compared the customer recency with the average diff_recency. If the value
exceeded the average, the customer was labeled a churner; otherwise, the customer was viewed as being
loyal. In total, we found that customers’ NPS score and purchasing patterns did not match across the three
years.
• While customer loyalty based on the NPS score classified customers into 70% promoters, 25% passives and
5% detractors, the customers’ purchasing behavior analysis shows that only 54% of customers were loyal
and 46% of the company’s customers were churners.
• In the Text-mining model every comment was analyzed automatically using the developed linguistic patterns
and then assigned to one of the seven high-level root-cause categories: communication, capability, parts,
price value, process adherence, quality and service capacity.
Predictive Analytics

15. service800.com Confidential
• The proposed customer loyalty predictive model showed how top churners are promoters in the NPS
category. The researchers selected 20 top churners based on their monetary indicator to evaluate their
customer loyalty. Customers with high spending levels are more likely to stay, and changing patterns in
spending may be an indicator of churn.
• The authors cross checked the results with the original NPS and it further confirmed that the single
measure is not a useful measure when calculating customer loyalty.
Top Churners vs. Sales Volume

16. service800.com Confidential
• A Bayesian network model is a probabilistic graphical model (a type of statistical model) that represents a set
of variables and their conditional dependencies via a directed acyclic graph (DAG).
• Using Baysian Network the researchers identified probabilities of loyal customers and churners specific to
geographical locations.
Baysian Network Customer Loyalty Probabilities
Probabilities
Loyalty
Status
Central
East
Central
West
South-
East
Ireland Scotland South-
West
North
Ireland
Churner 17% 16% 22% 1% 19% 22% 3%
Loyal 19% 20% 15% 4% 19% 17% 6%

17. service800.com Confidential
•This paper suggests companies to move away from single metric systems such as NPS to a nuance
multidimensional customer loyalty model that will help organizations to predict customer behavior.
•Though survey based measures are important they do not provide holistic view of all customer touchpoints
and practical insights into “why” customers are churning.
•The paper highlights the importance of comment questions and implementing appropriate text-mining model
to monitor the voice of the customer to better allocate company resources on churning customers.
•Furthermore the researchers shows the importance of capitalizing data points extracted from customer journey
to simultaneously measure the attitudinal and behavioral components of customer loyalty.
Conclusion

18. service800.com Confidential
• Mohamed Zaki,Dalia Kandeil, Andy Neely,Janet R.McColl-Kennedy, The Fallacy of the Net Promoter Score:
Customer Loyalty Predictive Model, 2016
• M. Scanagatta, C. P. de Campos, G. Corani, and M. Zaffalon. Learning Bayesian Networks with Thousands
of Variables. In NIPS-15: Advances in Neural Information Processing Systems 28, pages 1855–1863, 2015.
References