More Related Content Similar to Validation of Customer Survey (20) More from Business Over Broadway (14) Validation of Customer Survey1. Copyright Ā© 2010 Business Over Broadway
Validation of COMPANY Customer
Satisfaction Survey
Business growth through customer insight
Business Over Broadway
2. Copyright Ā© 2010 Business Over Broadway
ā¢ Executive Summary
ā¢ Components of Customer Feedback Programs
ā¢ Uses of Customer Feedback Data
ā¢ Company Surveys
ā¢ Definitions of Reliability and Validity
ā¢ Evidence of Reliability
ā¢ Evidence of Validity
ā¢ Summary
ā¢ Conclusions
ā¢ Recommendations
Overview
3. Copyright Ā© 2010 Business Over Broadway
ā¢ The company can feel confident using their customer
survey results when making business decisions
ā¢ Loyalty questions can be used to make reliable, useful
decisions to grow business through new and existing
customers
ā¢ Build customer-centricity in COMPANY by using loyalty
indices in:
ā executive dashboards, strategy/governance, decision making,
incentive compensation, benchmark against industry standards
ā¢ Proposed Recommendations
ā Change Customer Loyalty Measurement
ā Change Technical Support Measurement
ā Adopt best practices in Strategy/Governance, Business
Integration
Executive Summary
4. Copyright Ā© 2010 Business Over Broadway
ā¢ Adoption of best practices in each area ensures program success1
Components of Customer Feedback
Programs
1 In Hayes, B.E. (2009). Beyond the Ultimate Question: A Systematic Approach to Improve Customer Loyalty. Quality Press.
Milwaukee, WI. Loyalty Leading companies adopt specific business practices compared to Loyalty Lagging companies
5. Copyright Ā© 2010 Business Over Broadway
Employee, Partner and Customer Loyalty
Drive Business Results
Based on the book, The Service Profit Chain: How leading Companies Link Profit and Growth to Loyalty, Satisfaction and Value
(Heskett, Sasser & Schlesinger, 1997)
Employee Survey
Customer Survey
Partner Survey
Business Strategy
Business Programs
Company
PerformanceInternal
System
Employee
Satisfaction
Employee
Productivity
Perceived
Value
Customer
Satisfaction
Revenue
Profits
Employee
Loyalty
Customer
Loyalty
Partner
Advocacy
Purchasing
Retention Market
Share
6. Copyright Ā© 2010 Business Over Broadway
ā¢ Relationship Satisfaction Surveys
ā Focused on overall relationship with the customer
ā Delivery is not time-dependent (done at company discretion)
ā Includes Business Attribute questions and Loyalty-based questions
ā¢ Typically have ratings of Satisfaction
ā¢ Transactional Satisfaction Surveys
ā Focused on specific customer transaction
ā Delivery is time-dependent on transaction occurrence
ā Includes Business Attribute questions and Loyalty-based questions
ā¢ Typically have ratings of Satisfaction
ā¢ Targeted Surveys
ā Focused on drilling into specific areas for deeper insight
ā Delivery is on an as-needed basis
ā Compliment the Relationship and Transactional surveys
ā¢ Three survey types can be used in any given Account
Company Surveys
7. Copyright Ā© 2010 Business Over Broadway
ā¢ Customer surveys should result in reliable and valid
scores of customersā attitudes
ā Good measurement processes lead to good business decisions
ā¢ Reliability reflects the degree to which scores are
free from random error
ā Reliability deals with precision/consistency
ā Goal is to have a measurement system that
delivers reliable results
ā Reliability varies from 0 (no reliability) to 1
(perfect reliability)
ā¢ Validity reflects the degree to which
scores are measuring what they
are designed to measure
ā Validity deals with the meaning of the scores from the measurement system
ā Goal is to have a measurement system that delivers valid results
Reliability and Validity
ValidNot Valid
ReliableNotReliable
8. Copyright Ā© 2010 Business Over Broadway
ā¢ Four general forms of reliability
1. Inter-rater reliability1
ā Degree of agreement between two or more
raters who are rating same attribute
2. Test-retest reliability1
ā Degree of agreement between same
measure/rater over two time periods
3. Parallel-forms reliability
ā Degree of agreement between two
measures (A vs. B) that measure same thing
4. Internal consistency reliability1
ā Degree of consistency of ratings across
items within a measure
Reliability
=
Rater 1 Rater 2
=
Rater n Rater n
Attribute
1 Determination of reliability depends on number of questions in survey and types of surveys. Typically,
parallel-forms reliability cannot be studied as only one measure of customer satisfaction is available.
Time 1 Time 2
=
Rater n
Survey
Vers. A
Attribute
2
Attribute
m=
Survey
Vers. B=
Attribute
1
Rater n
9. Copyright Ā© 2010 Business Over Broadway
ā¢ Three approaches at establishing validity
1. Content-related approach
ā Examining the questions of the instrument to determine if they are a representative
sample of the universe of all possible questions
2. Criterion-related approach
ā Correlating scores from the instrument with some other criteria
ā¢ Based on some theory of the customer satisfaction construct
ā¢ Customer satisfaction scores should be correlated to other criteria (e.g., other
measures of Account quality)
3. Construct-related approach
ā Developing a nomological (lawful) network for the customer satisfaction scores
ā Framework/Theory/Model for understanding and using customer satisfaction scores
should be supported by data
Validity
11. Copyright Ā© 2010 Business Over Broadway
ā¢ Internal consistency determines extent to which
Contacts give similar ratings to items within a scale
ā Indexed by three indices
1. Item-total correlations - correlation between rating of single item
to aggregate of remaining items in the survey
2. Split-half correlation ā correlation between two halves of the
survey
3. Cronbachās alpha ā formula to calculate internal consistency
ā¢ Addresses the quality of the items in a scale
ā If reliability is high, ratings do not tend to vary across items
ā Expect some degree of internal consistency
ā¢ Do not expect perfect reliability ā Perfect reliability would indicate
no variability in ratings and no need to ask multiple questions
Internal Consistency Reliability
12. Copyright Ā© 2010 Business Over Broadway
Surveys
Contact 1
Contact 2
Contact n
Business
Attribute 1
Business
Attribute 2
Reliability Analysis
Internal Consistency Reliability
ā¢ Item-Total Correlation is the correlation of
specific attribute with aggregate of
remaining attributes
ā¢ Split-half reliability is the correlation
between average of the two halves of the
survey
ā¢ Cronbachās alpha (ļ”) estimate used to
determine internal consistency of ratings
(satisfaction) within a survey
ā¢ Calculated for any type of survey
that contains multiple questions
ā¢ Reliability varies from 0 (no internal
consistency) to 1 (perfect internal
consistency)
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
.
.
.
=
Rater n
Attribute
2
Attribute
m=Attribute
1
Data Model for Internal Consistency
Reliability Analysis
13. Copyright Ā© 2010 Business Over Broadway
Index (Measure) Definition Survey Questions
Customer Loyalty Index (4
questions)
Extent to which customers feel positively
toward COMPANY
Overall Satisfaction, Quality, Value, Recommend
Perceived Value of External
Information1 (7 questions)
Extent to which customers value external
sources of information regarding security
products
Security vendor websites, Whitepapers, Industry
analysts, Blogs / online communities, Security
publications, Peers, Webcasts / podcasts
Perceived Importance of
Doc/Mgmt/Reporting (3 questions)
Stated importance of Documentation,
Management, Reporting
Reporting, Management / Administration,
Documentation
Perceived Importance of Ease of
Use (3 questions)
Stated importance of user experience Ease of installation, End user experience, Ease of
upgrade
Quality of Security Product (8
questions)
Rating of quality of security product Security effectiveness, Reporting, Management /
Administration,
Documentation, Ease of installation, End user
experience, Ease of upgrade, Price
Quality of Technical Support (3
questions)
Rating of quality of technical support Technical support, Telephone, Email
Indices (Summated Scales)*
1 Items only appear in 2009 survey.
*Indices were created based on factor analyses of the survey questions; factor-analytic results show that specific survey items can be combined
together into one index. Survey questions that loaded highly on the same factor were combined into a single scale by averaging the responses
across items.
Factor analyses of survey items exposed several indices, each
measuring a general content area (e.g. loyalty, tech support).
14. Copyright Ā© 2010 Business Over Broadway
Internal Consistency Reliability by
Decision Influence
Decision-Making Role
INDEX
Entire
Sample
Final
Decision Influencer Purchasing
2009 Indices
1 Customer Loyalty Index* .89 .89 .89 .88
2 Perceived Value of External
Information
.85 .85 .83 .87
3 Perceived Importance of
Doc/Mgmt/Reporting
.80 .78 .79 .78
4 Perceived Importance of Ease of Use .79 .76 .80 .80
5 Quality of Security Products .91 .92 .91 .92
6 Quality of Tech Support .91 .91 .90 .91
2010 Indices
1 Customer Loyalty Index .82 .81 .83 .81
2 Perceived Importance of
Doc/Mgmt/Reporting
.70 .69 .66 .78
3 Perceived Importance of Ease of Use .66 .68 .64 .58
4 Perceived Quality of Security Products .89 .89 .88 .91
5 Perceived Quality of Tech Support .92 .93 .92 .95
Internal Consistency Reliability1
Major Points
ā¢ Factor analyses of survey
items exposed several indices,
each measuring a general
content area (e.g. loyalty, tech
support).
ā¢ Definition of index is based on
the aggregated items
ā¢ Internal consistency
reliabilities are acceptable and
similar across all decision-
making categories.
ā¢ Each index would be useful for
executive dashboards since
each provides reliable, useful
information.
1 No significance testing possible for Cronbachās alpha.
*See previous slide for description of each index and the survey items that represent each index.
15. Copyright Ā© 2010 Business Over Broadway
ā¢ Inter-contact reliability determines extent to which
Contacts within an Account give similar customer
satisfaction survey ratings
ā Indexed by a correlation between different Contacts within each Account across all
Accounts
ā¢ Inter-contact reliability addresses the consistency
of the implementation of the customer satisfaction
survey system across Contacts
ā Expect some degree of inter-contact reliability because Contacts are rating the same
Account
ā¢ Do not want perfect reliability ā Perfect reliability would indicate no utility of
surveying different management contacts within an organization
ā¢ Calculate inter-contact reliability for each survey
type across all business attributes
Inter-Rater Reliability
16. Copyright Ā© 2010 Business Over Broadway
Data Model for Inter-Rater Reliability
Analysis
Type of Contact
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Inter-Contact Reliability
ā¢ Correlation of satisfaction ratings between
different Contacts within Accounts
ā¢ Calculated for each type of customer survey
ā¢ Reliability could vary from 0 (no agreement
between Contacts) to 1.0 (complete
agreement between Contacts)
.
.
.
...
=
Rater 2
Attribute
Rater 1
17. Copyright Ā© 2010 Business Over Broadway
Inter-Contact Reliability (Decision-
makers vs. Influencers)
INDICES1
Decision
maker/
Influencer
1 Customer Loyalty Index* .19
2 Security Effectiveness -.12
3 Reporting .05
4 Management / Administration .15
5 Documentation -.04
6 Ease of Installation .16
7 End User Experience .02
8 Ease of Upgrade .22
9 Technical Support .05
10 Price .22
Major Points
ā¢ There is low agreement
between Contacts from different
decision-making standpoint.
ā¢ Contacts from different
decision-making standpoints
(e.g., Decision maker vs.
Influencer) show low agreement
in their ratings of COMPANY
ā¢ The degree of disagreement
across different survey
respondents within an Account
could reflect:
ā¢Poor communication
within an Account
ā¢Different expectations of
different functions
ā¢Different usage of
COMPANY products
1 Survey data are from 2010 Survey; N = 47 to 50. No statistically significant correlations.
*Customer Loyalty Index is composite of following four questions: Overall Satisfaction, Quality, Value
and Recommend. Functional job function includes Finance,
Not enough sample to examine āPurchasingā role.
18. Copyright Ā© 2010 Business Over Broadway
ā¢ Test-retest reliability determines extent to which
Contacts give similar ratings over time
ā Indexed by a correlation between ratings across two time periods
ā Higher correlation indicates that Contacts provide similar ratings over time
ā¢ Addresses the consistency of ratings over time
ā If reliability is high, relative ordering of ratings do not tend to vary over time across
different Accounts
ā Expect good degree of test-retest reliability because Contacts are rating the same
Account over time
ā¢ Contacts that are satisfied/dissatisfied at time 1 should also be
satisfied/dissatisfied at time 2
ā¢ Contacts with high/low value ratings at time 1 should also have high/low value
ratings at time 2
ā¢ Calculate test-retest reliability for satisfaction and
value ratings
Test-Retest Reliability
19. Copyright Ā© 2010 Business Over Broadway
Data Model for Test-Retest Reliability
Analysis
Time 1
Survey
Time 2
Survey
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Test-Retest Reliability
ā¢ Correlation of satisfaction ratings between
same Contacts across two time periods
ā¢ Correlation of satisfaction ratings (averaged
over Contacts) between same Accounts
across two time periods
ā¢ Calculated only for Surveys that are
repeated
ā¢ Reliability varies from 0 (no
agreement between time 1 and time
2) to 1.0 (complete agreement
between time 1 and time 2)
.
.
.
.
.
.
.. .
=
Contact n Contact n
Time 1 Time 2
20. Copyright Ā© 2010 Business Over Broadway
Test-Retest Reliability
INDICES/MEASURES
1 Customer Loyalty Index .69
2 Overall Satisfaction .32
3 Quality .58
4 Value .52
5 Recommend .70
6 Security Effectiveness .43
7 Reporting .53
8 Management / Administration .54
9 Documentation .44
10 Ease of Installation .50
11 End User Experience .49
12 Ease of Upgrade .45
13 Technical Support .51
14 Price .54
Test-Retest
Reliability1
1 Survey data are from 2009 (Time 1) and 2010 (Time 2). N = 472 to 478. All correlations significant at p < .01 level. CLI is a composite score
consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey
questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
=
Contact n Contact n
Time 1 Time 2
Major Points
ā¢ Test-retest reliability is high
ā¢ Customers tend to respond
in a similar way over two
time periods.
ā¢ Those who are satisfied
(dissatisfied) at time 1 tend
to be satisfied (dissatisfied)
at time 2.
22. Copyright Ā© 2010 Business Over Broadway
ā¢ Content validity is established when items in the
survey are a representative sample of some defined
universe of items
ā¢ Universe of survey items represents all items that
define the important elements of a companyās
performance
ā¢ Does the customer survey comprehensively assess
customer requirements?
ā¢ Must show ālinkā between set of items (survey) and
the universe of items
ā Link determined by Subject Matter Experts (SMEs)
Content-related Approach
23. Copyright Ā© 2010 Business Over Broadway
ā¢ Disloyal customers provide
comments regarding
improvements1
ā¢ Comments/issues range
from product to service issues
ā¢ All issues/reasons
covered by content of
customer survey
ā¢ Issues aligned with
results of driver analysis
Content-related Approach
1 Customer comments from those who rated Recommend question low.
24. Copyright Ā© 2010 Business Over Broadway
ā¢ Criterion-related validity relies on statistical
analysis rather than judgment
ā¢ Established when scores on the survey are related
to some other variable (criterion) already known
to measure similar attributes
ā¢ Key is developing/selecting right criteria to use in
validation study
ā Measures of Account Quality ā by Account Team
ā Objective Assessment Scores
Criterion-related Approach
25. Copyright Ā© 2010 Business Over Broadway
ā¢ Construct-related validity addresses what is being
measured by examining relationships between
constructs (e.g., customer satisfaction, customer
loyalty, other measures of Account quality)
ā¢ Concerned with theory and the proposed relationships
between constructs based on that theory
ā¢ Understanding the theoretical relationship of
constructs to other constructs
ā To what should customer satisfaction be (convergent validity)
and not be related (discriminant validity)?
ā¢ Need to examine patterns of relationships using the
survey and other instruments
Construct-related Approach
26. Copyright Ā© 2010 Business Over Broadway
2009 Mean SD N 1 2 3 4 5 6
1. Customer Loyalty Index 4.84 0.94 2941 0.89
2. Quality of Security Product 4.59 0.86 2917 0.77 0.91
3. Quality of Technical Support 4.81 1.09 2880 0.68 0.69 0.91
4. Perceived Value of External Information 4.34 0.97 2787 0.31 0.37 0.29 0.85
5. Perceived Importance of Doc/Mgmt/Reporting 4.77 0.87 2914 0.29 0.43 0.31 0.34 0.80
6. Perceived Importance of Ease of Use 5.05 0.87 2919 0.33 0.48 0.33 0.28 0.54 0.79
Correlations Among Indices
ļ§ Indices are correlated logically with each other
ļ§ Higher ratings on key indices (quality, value) correspond to higher
customer loyalty ratings compared to other indices (perceived
importance)
2010 Mean SD N 1 2 3 4 5
1. Customer Loyalty Index 5.54 1.27 3449 0.82
2. Quality of Security Product 4.72 0.75 3417 0.78 0.89
3. Quality of Technical Support 4.95 1.04 3387 0.67 0.65 0.92
4. Perceived Importance of Doc/Mgmt/Reporting 5.01 0.66 3434 0.11 0.21 0.12 0.70
5. Perceived Importance of Ease of Use 5.17 0.69 3437 0.10 0.23 0.12 0.41 0.66
1 Cronbachās alpha (internal consistency estimates) are located in the diagonal.
27. Copyright Ā© 2010 Business Over Broadway
ā¢ Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2009)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 4.84 2941
Understands Needs 4.57 2926 0.73
Addresses Needs 4.52 2924 0.78
Understands Better Than Competitors 4.30 2884 0.69
Security Effectiveness 5.18 2911 0.70
Reporting 4.40 2875 0.54
Management / Administration 4.52 2866 0.63
Documentation 4.23 2871 0.55
Ease of Installation 4.61 2884 0.58
End User Experience 4.66 2880 0.66
Ease of Upgrade 4.77 2856 0.61
Technical Support 4.80 2860 0.67
Price 4.35 2877 0.55
Compare COMPANY to Alternatives 4.06 2645 0.69
Tech Support via Telephone 4.96 1815 0.61
Tech Support via Email 4.79 1792 0.62
Tech Support via Web Knowledgebase 4.39 2102 0.49
1 No significance testing possible for Cronbachās alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
28. Copyright Ā© 2010 Business Over Broadway
ā¢ Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2010)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 5.54 3449
Understands Needs 4.32 3414 0.44
Addresses Needs 4.31 3408 0.48
Understands Better Than Competitors 4.37 3337 0.52
Security Effectiveness 5.21 3407 0.65
Reporting 4.58 3371 0.54
Management / Administration 4.67 3387 0.67
Documentation 4.48 3372 0.54
Ease of Installation 4.72 3382 0.56
End User Experience 4.73 3371 0.66
Ease of Upgrade 4.78 3356 0.59
Technical Support 4.91 3360 0.66
Price 4.62 3348 0.47
Compare COMPANY to Alternatives 4.01 3020 0.77
Tech Support via Telephone 5.05 2408 0.63
Tech Support via Email 4.91 2337 0.64
Tech Support via Web Knowledgebase 4.47 2585 0.55
1 No significance testing possible for Cronbachās alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
29. Copyright Ā© 2010 Business Over Broadway
ā¢ Use previous validity approaches (content and
criterion) to support a theoretical model of customer
satisfaction
ā¢ Various theoretical models
state that customer loyalty
is impacted by satisfaction with
product and satisfaction with
service
ā¢ Need to examine the pattern of
relationships among these
constructs
ā¢ Pattern of correlations should support this model
Construct-related Approach
Service
Satisfaction
Product
Satisfaction
Customer
Loyalty
Basic Theory of
Customer Satisfaction
30. Copyright Ā© 2010 Business Over Broadway
ā¢ Test the model by examining
correlations among three
variables:
ā Product Quality (Quality of Security Eff.)
ā Tech Support Quality
ā Customer Loyalty (CLI)
ā¢ Path analysis1 applied to
observed correlations
ā Isolates degree of impact a given
variable has on other variables
ā¢ Analysis supports theory that
both product quality and tech
tech support quality impact
customer loyalty independently
Construct-related Approach
CORRELATIONS
CLI
Tech
Qual
Product Quality .70 .53
Customer Loyalty (CLI) - .68
1 For detailed description of path analysis methodology, please see: Pedhazur, E. J. (1982). Multiple
regression in behavioral research: Explanation and prediction (2nd Ed.) Holt, Rinehart and Winston.
New York, NY.
Tech Support
Product
Effectiveness
Customer
Loyalty
Path Analysis
.11
.54
.51
31. Copyright Ā© 2010 Business Over Broadway
ā¢ Support Level and Customer Loyalty
ā¢ Three levels of Support (Standard, Premium, Platinum)
ā¢ Lower customer loyalty reported by customers with
Platinum support
ā Platinum support customers could have higher expectations than
Standard support customers.
Support Level
32. Copyright Ā© 2010 Business Over Broadway
ā¢ Difference found across regions1
ā APAC has highest levels of customer loyalty and quality ratings
ā Southern Europe as lowest levels of customer loyalty and quality
ratings
Segmentation By Region
1 Based on survey results from 2010.
33. Copyright Ā© 2010 Business Over Broadway
Segmentation By Industry
Difference found across industries1
HED has the lowest levels of customer loyalty
1 Based on survey results from 2010.
34. Copyright Ā© 2010 Business Over Broadway
Segmentation By License Tier
Difference found across license tiers1
License tiers 3 and 4 have highest levels of customer loyalty and quality ratings
1 Based on survey results from 2010.
35. Copyright Ā© 2010 Business Over Broadway
ā¢ Internal consistency reliability
ā Contacts tend to give similar ratings across the different items of the same
scale
ā¢ Non-perfect reliability indicates that different items still provide different
information about customersā attitudes
ā¢ Reasons for extremely high reliability
ā High degree of redundancy of items
ā Rater fatigue due to survey length so respondents rate all questions similarly
ā¢ Inter-contact reliability
ā Contacts within Accounts show little agreement in ratings
ā Different types of Contacts provide different vantage point when rating the
company
ā¢ Poor communication within Account across levels and/job functions
ā¢ Different Contacts have differing expectations
ā¢ Contacts are treated differently by Account team depending on their
level/function
Summary of Reliability of Survey Ratings
36. Copyright Ā© 2010 Business Over Broadway
ā¢ Content-related approach
ā Questions in customer survey are representative sample
of universe of questions
ā¢ Items comprehensively cover customer requirements
throughout all phases of the customer lifecycle
ā¢ Dissatisfied customersā open-ended comments are addressed
by questions in the survey
ā¢ Criterion-related approach
ā No external criteria are available at this time
ā¢ Construct-related approach
ā Pattern of relationships of customer satisfaction data
supports theory of customer satisfaction
ā¢ Customer satisfaction is related to customer loyalty
ā¢ Product quality impacts loyalty through perceived value
Summary of Validity
37. Copyright Ā© 2010 Business Over Broadway
ā¢ Self-assessment survey identified strengths and areas
needing improvement1
ā¢ Overall, low to moderate adoption rates of best practices
across all components of customer feedback programs
ā Customer feedback not as important as financial metrics,
customer feedback not integrated with CRM system, problem
resolution not integrated into CRM system
ā¢ Strengths seen in specific areas of COMPANYāS customer
feedback program:
ā Top executive is champion of the program, multiple methods of
data collection, Web surveys, various measures of customer
loyalty, customer research presented externally, existing
customer data used for segmentation
Customer Feedback Programs Best
Practices Survey
1 Survey identifies extent to which company adopts best practices in their customer feedback program. EMPLOYEE JANE DOE
completed the online survey (http://www.businessoverbroadway.com/cfpbpi.htm) for COMPANY on June 10, 2010. See survey for
questions.
38. Copyright Ā© 2010 Business Over Broadway
ā¢ The companyās survey process produces reliable,
valid indices of customersā attitudes
ā¢ Evidence of reliability
ā All indices are highly reliable ā items within indices
can be combined to create summary scores
ā¢ Evidence of validity
ā Survey questions comprehensively assess important
areas to the customers
ā Each index measures unique construct apart from the
other indices
ā Relationships among indices supports customer
satisfaction models
Conclusions
39. Copyright Ā© 2010 Business Over Broadway
ā¢ The company can feel confident using customer
survey results when making business decisions
ā Identifying drivers of customer loyalty
ā Using Indexes to manage customer relationships
ā¢ Understand why different customers within the
same Account hold different attitudes toward
COMPANY
ā Do they communicate with each other (through
Company-sanctioned User groups or individually)? They
could get together and talk about the benefits of
COMPANY. Could COMPANY assist in this process?
Conclusions
40. Copyright Ā© 2010 Business Over Broadway
ā¢ Changes in Customer Loyalty Measurement
ā Put loyalty question at start of survey (to assess top of mind
responses) with common rating scale (recommend 0 to 10)
scale)
ā Add questions to measure other components of customer
loyalty
ā¢ Purchasing Loyalty: Example: How likely are you to purchase
additional/different types of products from COMPANY?
ā¢ Retention Loyalty: Example: How likely are you to switch to a different
security provider within the next 12 months?
ā¢ Benefits
ā Make decisions to grow business through new and existing
customers
ā Loyalty indices can be used to build customer-centricity
ā¢ Executive dashboards, guide strategy/governance, resource
allocation, incentive compensation, benchmark against industry
standards
Recommendations on Method
41. Copyright Ā© 2010 Business Over Broadway
ā¢ Changes in Technical Support Measurement
ā Move all technical support questions into one section
ā¢ Telephone, Email, and technical support
ā Consider adding other technical support questions to
broaden technical support coverage
ā¢ Knowledge, professionalism, care
ā¢ Benefits
ā Easier for customers to complete survey
ā Provide more specific feedback to technical support
personnel
ā¢ Consider ways to increase response rates
ā Email invitation content
Recommendations on Method
42. Copyright Ā© 2010 Business Over Broadway
ā¢ Integrate customer feedback into companyās
strategy/governance1
ā Companyās strategic vision, mission and goals; executivesā
objectives and incentive compensation, front-line employeesā
objectives and incentive compensation
ā¢ Integrate customer feedback into daily business
processes1
ā CRM system; company/executive dashboards; communicate
results regularly to the entire company
ā¢ Conduct in-depth research using customer survey results1
ā Linkage analysis between customer feedback and operational
metrics, financial metrics, employee metrics.
Recommendations on Strategy,
Integration and Research
1 Based on Customer Feedback Programs Self-Assessment Survey Results.
43. Copyright Ā© 2010 Business Over Broadway
ā¢ Consider other āfinancialā metrics to track and link to
customer survey
ā Consult with CFO to consider other financial metrics that are
tracked and consider using those to identify which aspects of
customer satisfaction/loyalty relate to financial growth of
COMPANY
ā¢ e.g. Renewals
ā Track financial metrics over time in conjunction with the
customer survey to conduct longitudinal studies
ā¢ Study impact of attitudes at time 1 on financial metrics at time 2 and
beyond
ā¢ Linkage study between COMPANYāS transactional survey
and relationship survey
ā To what extent do transactions (e.g. in call center) impact long-
term customer loyalty
Recommendations for Future Research
44. Copyright Ā© 2010 Business Over Broadway
For More Information
Business growth through customer insight
Business Over Broadway
Bob E. Hayes, Ph.D.
Email: bob@businessoverbroadway.com
Web: www.businessoverbroadway.com
Blog: www.businessoverbroadway.blogspot.com
Twitter: www.twitter.com/bobehayes