This document describes a new approach of Net Promoter Scoring ranking which uses Issue Resolution Rate to determine true ranking over whole population using Binomial Statistical Distribution instead of relying on actual scoring values based on a sample of Customers. Issue Resolution is factual and owned by the Company while actual NPS is based on Customer Empathy.

1. Mai Dang, Telstra, November 11th
2016 1 of 19
A New NPS Benchmarking Process with
2016 US Election Interpretation
Mai Dang
Telstra Corporation Limited
11th
November 2016
Net Promoter Scoring (NPS) process consists of two sides:
1. Company’s effort to maximize its Products and Services Performance measured by “Issue
Resolution” metric. A “Resolution Yes” is when Customer answering “Yes” to the question
“Has your issue been resolved?”
Resolution Rate “Yes” (RR) over the survey volume is the most often used. Other answers
like “No” or “Too early to tell” are grouped as 1-RR. RR values are between 0 and 1.
2. Customer’s perception of this effort is the NPS scoring. NPS is narrowed down to 3
Categories according to scores between 0 and 10:
o Advocate (score 9-10) are loyal promoters who will keep buying and refer others.
o Passive (score 7-8) are satisfied but are vulnerable to churn due to different reasons.
o Detractor (score 0-6) are unhappy customers who can damage the brand through
negative word-of-mouth.
NPS is calculated based on difference between Advocate and Detractor in percentages of survey
volume. NPS values are between -100 and +100.
Current NPS benchmarking process [5] is based solely on NPS value. However Customer experience
is more than just NPS as the general Customer feeling is based on issue that was resolved or not.
Issue Resolution rate is a factual and objective metric as it reflects the Company Advocacy’s effort.
Issue Resolution rate determines the true NPS using Binomial Distribution with the whole population
over the reporting period whilst the actual NPS feedback only valid for a sample population.
The tips given in addition to the bill amount in a Restaurant is a good analogy with NPS. If the
services are received as either “very bad” or “very good” then the tips are consistent and variations
are small between payments. The tip amount is the NPS score and represents Customer Empathy
towards the Restaurant Service which is Resolution Rate.
Analogy with Restaurant stops here as NPS context is more intricate than restaurant payment as the
tips and bill amounts in NPS are collapsed into one figure which is NPS. To work out Customer
Empathy, Resolution Rate needs to be included in the process with the use of Binomial Distribution
where NPS values are calculated from the Resolution Rates.
This current exercise proposed a new NPS benchmarking process with a simultaneous ranking on
both NPS and Resolution Rate. If NPS ranking is consistent with Resolution Rate’s then both existing
and new NPS benchmark report the same outcome. If the rankings between NPS and Resolution
Rate are inconsistent or overlapping then the new benchmark outcome will be contradicting the
existing ranking.

2. Mai Dang, Telstra, November 11th
2016 2 of 19
The results of 2016 US Election [7] with Poll and Vote data demonstrate the use of the new
benchmarking process with Empathy weigh leans toward 1 for the Vote data. Poll data as a
snapshot, has Empathy weights of 0.95 and 1.05 for Republicans and Democrats respectively.
The 2014 NPS Benchmark between Apple and Samsung for Smartphones products [5] is another
example that shows an equivalence with US Election Poll data as both are just NPS snapshots.
To have a robust NPS Benchmarking system, Empathy weight equal to 1 must be used as base for
NPS ranking across the board.
There are three parts of the discussion:
 Part I: Provide a method to work out a NPS score for a given Resolution “Yes” with a neutral
Empathy (weight=1), either by:
a. Using Binomial Distribution algorithm with RR as an input to the model.
b. Using “Bean machine” process in conjunction with 11 row calculation of NPS from
Pascal’s triangle algorithm to perform NPS calculation from RR as an input.
Method (a) can be implemented easily within a spreadsheet. For this exercise, the
results are obtained using approach (b) with R as programming language. A simulation
prototype written Java demonstrates the approach (b) is also discussed using a random
number between 0 and 1 to simulate a Resolution “Yes” or not equal “Yes”.
 Part II: NPS Life Cycle revealed the Strategy to improve the Customer Experience.
 Part III: Introduction of Customer’s Empathy to the NPS actual data. The results are consistent
between Human led and Technology based channels or between Residential and Business
segments and even the timing of NPS process (online and episode) also had a driving effect on
how the score was issued.
Empathy according to [1] is defined as “generally includes responding to positive affects as well as
negative ones without, however, necessarily requiring doing anything about it”.

3. Mai Dang, Telstra, November 11th
2016 3 of 19
I.NPS machine
1. Bean machine
The easy way to explain analytically the NPS process is to use a board game called “Bean machine”
with a demo from you-tube [2].
From a middle and narrow top entrance of the “Bean machine”, the beans were introduced and
fallen into the bottom slots (see Figure 1) through the gaps between the wooden ticks. Let’s assume
there was a block of 11 x 11 wooden sticks on the board. Each bean will hit these ticks on the way
down to the bottom slots. These ticks constitute the core of Customer interaction with the company.
The 7 leftmost bottom slots are dedicated for Detractors. The 2 rightmost slots are for Advocates.
The 2 slots in the middle are for Passive Customers.
If there are no intervention to the way the wooden ticks are setup, the chance the beans fallen to
right or to the left of the tick is 50% each. With NPS, more the beans fallen to the right better the
NPS is, hence all wooden ticks will have to be “tampered” which reflects the true essence of
Company’s effort in ethical terms which is measured as the Resolution Rate.
Figure 1: “Bean machine” from you-tube [2] https://www.youtube.com/watch?v=3m4bxse2JEQ
2. Pascal’s triangle
Pascal’s triangle calculation [3] was used to work out how many beans will be collected at 11 bins at
the bottom of the board as below. Pascal’s triangle principle is explained as figure 2.
Figure 2: Pascal’s triangle calculation. See Reference [3]

4. Mai Dang, Telstra, November 11th
2016 4 of 19
3. Bean machine simulation through Pascal’s triangle and Binomial Distribution
Table 1: Pascal’s triangle applied to NPS scoring with values at row 10 as Binomial Coefficients
Binomial Distribution is the statistical expression of the chance for two outcomes (like “Head” and
“Tail” of coin tossing) through a sequence of consecutive events which are linked to Pascal’s triangle
with also two outcomes (“left” and “right” direction) and number of events called Rows from Tables
1 and 2.
The “Bean machine” provides an idea and Pascal’s triangle, Binomial Distribution [4] provides a
framework for building of what is called a “NPS machine” with different adjustments of the wooden
ticks which are “Resolution Yes” in NPS terminology.
The results at the Rows 10 applied to NPS from the Table 2 are used to calculate the chance of each
NPS score using Binomial algorithm as below
1. Detractor score 0: (1-RR)^10
2. Detractor score 1: 10*(1-RR)^9 *(RR)^1
3. Detractor score 2: 45*(1-RR)^8 *( RR)^2
4. Detractor score 3: 120*(1-RR)^7 *(RR)^3
5. Detractor score 4: 210*(1-RR)^6 *( RR)^4
6. Detractor score 5: 252*(1-RR)^5 *( RR)^5
7. Detractor score 6: 210*(1-RR)^4 *( RR)^6
8. Passive score 7: 120*(1-RR)^3 *( RR)^7
9. Passive score 8: 45*(1-RR)^2 *( RR)^8
10. Advocate score 9: 10*(1-RR)^1 *( RR)^9
11. Advocate score 10: 1*(RR)^10

5. Mai Dang, Telstra, November 11th
2016 5 of 19
Table 2: Normalised NPS scoring with total of each row equal to 1 at a Resolution “Yes” 50%
The simulated scores at the bottom row of table 2 can be implemented within an excel spreadsheet.
Another method to calculate NPS from RR is to use a programming language like R without explicitly
using Binomial expressions which is discussed at next section 3.
4. Bean machine simulation through random number programming using Java.
There is another approach from calculating NPS from RR instead of using Binomial Distribution, is to
use a random number between 0 and 1 is generated each time the bean from the “Bean machine”
hits the wooden tick. If this random number is less than the value of Resolution “Yes” (RR) then the
bean will fall to the right, towards “Advocacy” end. If the random number is between value of RR
and 1 then the bean will fall to the left, towards the “Detractor” end.
This process is encapsulated in a java written application. The results are shown from the Figures 2
and 3 using RR as 50% and NPS is -81 after 10,730 surveys taken. This NPS -81 is confirmed by
Binomial algorithm approach from Table 2 with a value -82 which is good enough to be used.

6. Mai Dang, Telstra, November 11th
2016 6 of 19
Figure 2 : Logical “Bean Machine” NPS simulation using Java with Resolution “Yes” at 75
Figure 3 : NPS Histogram with Resolution Yes 50% after 10,730 surveys
At RR 50%, the highest outcome according to Figure 3 is Detractor with scores 5, 4 and 6. Passive
score 7 is fourth. Advocate score 9 and 10 occupied the last ranking.

7. Mai Dang, Telstra, November 11th
2016 7 of 19
5. NPS generation from Resolution “Yes” as an input, using R
Figure 3: Implementation of Pascal’s triangle algorithm using R codes
With Empathy index 1, NPS values are generated over 100 values of RR “Yes” between 0 and 1 are
shown from the figure 5 using the Pascal’s triangle algorithm with R codes as per Figure 4.
R coding:
Figure 4: R codes to generate NPS profile versus Resolution “Yes”

8. Mai Dang, Telstra, November 11th
2016 8 of 19
Figure 5: NPS profile versus Resolution Rate “Yes”
II. NPS Life Cycle
Figure 7 shows the NPS life cycle from the value -100 going through the median at 0 and
terminating at value 100. The birth of NPS at -100 is with 100% Detractor.
As NPS climbed up to the median value 0, Detractor population is converted to Passive at
same time Passive itself is converted to Advocate. At the left hand side of the median from
Figure 7, the production of Passive from Detractor is much faster than its consumption to
produce Advocate. The profile of Passive rising up to a maximum of 50% at NPS 0. This
dynamic interpretation is proven by the rising profile of Passive because if the consumption
of Passive toward Advocate is fast and instant then Passive profile would be closed to 0%
and flat.
From NPS values going from 0 towards 100, the speed of consumption of Passive to produce
Advocate overtakes its production from Detractor, hence its profile is dropping slowly
toward 0 and so Detractor when NPS reaches its value +100.
These observations from the Figure 7 implies the fact that Advocate doesn’t come from
Detractor but Passive. Passive itself comes from both Detractor and Advocate in a reversible
passage.

9. Mai Dang, Telstra, November 11th
2016 9 of 19
Strategy for NPS improvement depends on current location of NPS. To increase NPS, either
Detractor would have to diminish or Advocate to increase.
From Figure 7, at NPS equal - 37 both Detractor and Passive are equal in population 45%
with Advocate only 10%. An improvement of NPS will have to be from the consumption of
Detractor to produce Passive.
At NPS 0, both Detractor and Advocate are at 23.5% and Passive at 53%. A conversion of
Passive to Advocate is starting to be more effective than Detractor to Passive.
At NPS +37 where Passive and Advocate are equal at 45% and Detractor is only 10% then
Passive are the one Companies must listen to as Passive conversion to Advocacy is more
efficient.
R coding:
Figure 6: R codes to display NPS Life Cycle from Figure 7
Figure 7: NPS Life Cycle of Detractor, Passive and Advocacy
Conversion of Passive to Advocacy is
faster than Detractor to Passive
Conversion of Detractor to Passive
is faster than Passive to Advocacy

10. Mai Dang, Telstra, November 11th
2016 10 of 19
III. NPS with Ethical Empathy
1. Design of Customer Empathy Template
To include Empathy weight, NPS is recalculated with RR multiplied by the Empathy weight and
the results with different weights from 0.8 to 1.35 are shown in Figures 8 and 9. “Empathy” zone
with green lines where weight is greater than 1. “Less Empathy” zone with red lines and weights
are less than 1. The bold green line in the middle is with weight 1 where the correct NPS is
calculated for a given RR. This is the bill amount for the restaurant analogy discussed in section II
earlier.
1.1 Use case with constant Resolution Rate Yes
“Empathy” from Figure 8 implies the fact for a given RR says 80%, NPS calculated is +26 with
Empathy weight of 1 and when the weight goes up to 1.2, NPS is rising to +94. NPS drops to -44
when Empathy weight fell to 0.8. Empathy in this example involves an increase of NPS for a
constant RR 80%.
For Empathy weight 1.2, a typical Customer thinking is “I likes what you are doing even though
your resolution has not been improved compared to the past”.
For Empathy weight 0.8, a typical Customer thinking is “As my issue has not been resolved today
even though your service level is the same compared to the past, I can’t give you the same
score”.
Figure 8: Customer Empathy for a constant Resolution “Yes”

11. Mai Dang, Telstra, November 11th
2016 11 of 19
1.2 Use case with constant NPS
From figure 9, with a constant NPS at 0, value a RR equal to 75% with Empathy weight of 1. If
the weight goes up to 1.25, RR drops back to 60% or if the weight goes down to 0.8, RR
increases to 94%. “Empathy” here involves a drop of RR for same NPS (“I likes what you are
doing and I give you same NPS score even though your resolution has not been as good as
other channel”).
For Empathy weight 1.25, a typical Customer thinking is “I likes what you are doing even
though your resolution (60%) has not been as good as other channels. Instead of giving you a
negative NPS, l give a better NPS”
For Empathy weight 0.80, a typical Customer thinking is “Even though your overall service
level has increased to 94%, I am not comfortable with your resolution process hence give the
same NPS score”.
If the mass of positive thinking customers are more than the negative one, the Empathy at
the end of the day would define the channel Customer Service.
Figure 9: Customer Empathy for a constant NPS
2. Implementation of Empathy
The R codes from Figure 3 are now redesigned by including Empathy weights
between 0.8 and 1.35 are as per Figure 10 below. The final data frame now contains
one extra column Empathy weight and NPS is recalculated each time Empathy
weight changes.
For the method that used Binomial expressions, the equations are now including Empathy
weight with RR ranging from 0 to either 1/Empathy or 1 depends which one reaching 1 first:

12. Mai Dang, Telstra, November 11th
2016 12 of 19
1. Detractor score 0: (1-RR*Empathy)^10 with RR ∈ {0, maximum(1/Empathy, 1)}
2. Detractor score 1: 10*(1-RR*Empathy)^9 *(RR*Empathy)^1
3. Detractor score 2: 45*(1-RR*Empathy)^8 *( RR*Empathy)^2
4. Detractor score 3: 120*(1-RR*Empathy)^7 *(RR*Empathy)^3
5. Detractor score 4: 210*(1-RR*Empathy)^6 *( RR*Empathy)^4
6. Detractor score 5: 252*(1-RR*Empathy)^5 *( RR*Empathy)^5
7. Detractor score 6: 210*(1-RR*Empathy)^4 *( RR*Empathy)^6
8. Passive score 7: 120*(1-RR*Empathy)^3 *( RR*Empathy)^7
9. Passive score 8: 45*(1-RR*Empathy)^2 *( RR*Empathy)^8
10. Advocate score 9: 10*(1-RR*Empathy)^1 *( RR*Empathy)^9
11. Advocate score 10: 1*(RR*Empathy)^10
Figure 10: Empathy Implementation within NPS model using R codes
3. Where NPS actual data fit within Empathy template
Weekly aggregates of NPS and Resolution Rate Yes results are compiled against the
calculated NPS profiles of Resolution Rate Yes. Due to Commercial Sensitivity, the channel
names have been withdrawn.
The approach to measure actual Empathy is to calculate the minimum distance between
actual NPS and Resolution Rate Yes coordinator point to one of the Empathy weights
between 0.8 and 1.35. The closest weight will have a minimum of distance. Empathy weight

13. Mai Dang, Telstra, November 11th
2016 13 of 19
variation is quite small (0.05) hence a maximum of error would be 5% on the Empathy
weight measured.
Residential and Business Customer segments from both Human Dialog and Technology
based channels are measured from table 2.
Table 2: Empathy Index measurement.
Figures 12 and 13 compare between Human Dialog and Technology based channels for
Residential and Business segments. The Empathy negative in Technology channel is only
observed in Retail and not in Business segment. This reversed effect in Empathy could be
due to many factors as the objectives of these Technology channels between Residential and
Business segments are quite different to each other.
With another Residential Customer Product from its Online and Episode channel, Empathy is
found more favourable when NPS was issued by Customer immediately after the transaction
than what was provided at some time later with Episode channel (see figure 14).
Figure 12: NPS versus Resolution Rate Yes for Residential Channels

14. Mai Dang, Telstra, November 11th
2016 14 of 19
Figure 13: NPS versus Resolution Rate Yes for Business Channels
Figure 14: NPS versus Resolution Rate Yes for a Residential Online and Episode Channel

15. Mai Dang, Telstra, November 11th
2016 15 of 19
The observed facts from figures 12, 13 and 14 above led to 3 concluding remarks
a. Human led channel received more Empathy than a technology based channel from
Residential Customers.
b. Business Customers tend to be more generous in Empathy than its Residential
counterparts.
c. Customers when asked to provide NPS online, have tendency to have more Empathy
than same question was asked some time later.
As a general guideline, if Customer Service is either improved or deteriorated then there
must be a follow up of NPS score accordingly for a neutral NPS scoring process (Empathy 1).
Reality is Customers don’t know what correct NPS for current experience level or cannot
remember what the previous experience was like to make a decision. Empathy plays an
important effect each time there is a variation in Customer Service expressed under
Resolution Rate Yes. If an instant NPS score calculated for Service Level for the day with an
Empathy equates 1, is displayed online, then there would be more chance that all channels
would converge to a neutral Empathy NPS.
IV. 2014 NPS Benchmark
Figure 15: A new NPS benchmark with overlapping between NPS and Resolution Rate
rankings
Empathy has a profound and changing effect on NPS benchmark with a hypothetical
situation where Company A has a NPS +67 higher than its competitor at NPS +54. From
traditional NPS benchmark, A’s Customer Experience is ranked first. However if Resolution
Rate and Empathy calculation are included in the benchmark as figure 15 shown and NPS is
taken at the projection of Empathy weight 1 then A’s NPS is now at around +25 which is
+25
+75Company A
(e.g. Apple NPS 2014 +67)
A’s Competitor
(e.g. Samsung NPS 2014 +54 )

16. Mai Dang, Telstra, November 11th
2016 16 of 19
lower than its competitor with a corrected NPS at around +75 as shown from figure 15. With
a neutral Customer Empathy, A’s competitor is ranked first and A is in second.
A’s Customer Empathy at 1.1 higher than its competitor’s at 0.95 even though A’s Resolution
Rate 0.8 is lower than its competitor at 0.9. This underlined the fact that A’s Customer
Service is better than its competitor even though A’s issue resolution rate is lower.
A’s competitor resolves more issues than A and yet Customers expressed more Empathy to
A because of something A’s competitor doesn’t have. A’s competitor must look into its
Customer Service process to see where it failed in Customer Empathy.
This hypothetical scenario could have been the real situation for 2014 NPS Benchmark
results in Smartphones section between Apple (Company A) and Samsung with NPS at +67
and +54 respectively [5].
Figure 16: A new NPS benchmark with consistent NPS and Resolution Rate rankings
If Apple Resolution Rate is higher than Samsung’s with either the red line is now relocated to
the left of the Apple green line (see figure 16) or Apple’s green line is relocated to the right
of Samsung’s red line (see figure 17) then existing and new benchmark report the same
outcome with the new NPS projection at a neutral Empathy (weight=1).
Outcomes from figures 16 and 17 are now Apple and Samsung NPS are either +25 and +5 or
+95 and +75 respectively, which are consistent with current NPS ranking with Apple first and
Samsung second (Apple +67 and Samsung +54).
Company A
(e.g. Apple NPS 2014 +67)
A’s Competitor
(e.g. Samsung NPS 2014 +54 )
+25
+5

17. Mai Dang, Telstra, November 11th
2016 17 of 19
Figure 17: A new NPS benchmark with consistent NPS and Resolution Rate rankings
V. 2016 US Election Benchmark
NPS is calculated between 2 parties Democrat and Republican with results taken from
National Polling Average [6] and Official 2016 US election results taken on the day after the
vote as below:
% Voters at the Poll NPS
Democrat = 45.5 % (45.5 – 42.2) / 100 = 3.3
Republican = 42.2 % (42.2 – 45.5) / 100 = -3.3
Electoral Votes at the Vote NPS
Democrat = 228 Electoral Votes (228 – 290) / (228+290) = + 12
Republican = 290 Electoral Votes (290 – 228) / (228+290) = + 12
Resolution Rate on x-axis is replaced by “% Issues promised to be resolved”.
Company A
(e.g. Apple NPS 2014 +67)
A’s Competitor
(e.g. Samsung NPS 2014 +54 )
+75
+95

18. Mai Dang, Telstra, November 11th
2016 18 of 19
Figure 18: US 2016 Election results with NPS Benchmarking
Democrat as incumbent party by definition is not in a strong position to have “% Issues
promised to be resolved” higher than the challenger party, as a result Republican appears on
the right of the template similarly to the situation described between Apple and Samsung
from figure 15.
“% Issues promised to be resolved” is allocated at 0.72 for Democrat and 0.77 for Republican
according to figure 18.
The polling data is only a snapshot of voters with Empathy weights between 0.95 and 1.05.
Republicans have a higher “Issue promised to be resolved” (i.e. resolution rate) but poorer in
Empathy weight and % Voters (e.g. NPS) than Democrats before the vote.
Election data has a much larger population with Empathy weight leans towards 1.
Republican with its “% Issues promised to be resolved” higher won the election. Similar
interpretation for Samsung from the figure 15.
This adaptation exercise reinforces the use of Issue Resolution Rate in conjunction with NPS
at a neutral Empathy (weight=1) for NPS Benchmarking process.
NPS actual data is only a snapshot with Empathy weights fluctuate over a broad range of
Empathy weights hence the existing benchmarking process lacks of foundation for a correct
benchmarking. The only way to align all actual NPS snapshot results is to add Resolution
Rate to the NPS template and vertically project the NPS to the Empathy weight 1 curve.
The value allocation of “% Issues promised to resolve” is a key step to align the NPS at the
Poll and at the Vote on a vertical line where the Vote NPS will be located at the Empathy
weight equal to 1. If “% Issues promised to resolve” is well formulated as per Allan
Lichtman’s 13 keys [8] then this approach could be used for prediction of % Voters or NPS
on the election day.

19. Mai Dang, Telstra, November 11th
2016 19 of 19
Reference
[1] http://www.iep.utm.edu/emp-symp/,
[2] https://www.youtube.com/watch?v=3m4bxse2JEQ
[3] http://www.mathsisfun.com/pascals-triangle.html
[4] http://en.wikipedia.org/wiki/Binomial_coefficient#Binomial_coefficient_with_n.3D1.2F2
[5] https://customergauge.com/news/2014-net-promoter-benchmarks/
[6] http://www.usatoday.com/pages/interactives/2016/election/poll-tracker/
[7] https://pollyvote.com/en/components/index-models/keys-to-the-white-house/
[8] “Predicting the Next President: The Keys to the White House 2016 ed. Edition”
by Allan Lichtman ISBN-13: 978-1442269200 ISBN-10: 1442269200