SlideShare a Scribd company logo
Copyright © 2010 Business Over Broadway
Validation of COMPANY Customer
Satisfaction Survey
Business growth through customer insight
Business Over Broadway
Copyright © 2010 Business Over Broadway
• Executive Summary
• Components of Customer Feedback Programs
• Uses of Customer Feedback Data
• Company Surveys
• Definitions of Reliability and Validity
• Evidence of Reliability
• Evidence of Validity
• Summary
• Conclusions
• Recommendations
Overview
Copyright © 2010 Business Over Broadway
• The company can feel confident using their customer
survey results when making business decisions
• Loyalty questions can be used to make reliable, useful
decisions to grow business through new and existing
customers
• Build customer-centricity in COMPANY by using loyalty
indices in:
– executive dashboards, strategy/governance, decision making,
incentive compensation, benchmark against industry standards
• Proposed Recommendations
– Change Customer Loyalty Measurement
– Change Technical Support Measurement
– Adopt best practices in Strategy/Governance, Business
Integration
Executive Summary
Copyright © 2010 Business Over Broadway
• Adoption of best practices in each area ensures program success1
Components of Customer Feedback
Programs
1 In Hayes, B.E. (2009). Beyond the Ultimate Question: A Systematic Approach to Improve Customer Loyalty. Quality Press.
Milwaukee, WI. Loyalty Leading companies adopt specific business practices compared to Loyalty Lagging companies
Copyright © 2010 Business Over Broadway
Employee, Partner and Customer Loyalty
Drive Business Results
Based on the book, The Service Profit Chain: How leading Companies Link Profit and Growth to Loyalty, Satisfaction and Value
(Heskett, Sasser & Schlesinger, 1997)
Employee Survey
Customer Survey
Partner Survey
Business Strategy
Business Programs
Company
PerformanceInternal
System
Employee
Satisfaction
Employee
Productivity
Perceived
Value
Customer
Satisfaction
Revenue
Profits
Employee
Loyalty
Customer
Loyalty
Partner
Advocacy
Purchasing
Retention Market
Share
Copyright © 2010 Business Over Broadway
• Relationship Satisfaction Surveys
– Focused on overall relationship with the customer
– Delivery is not time-dependent (done at company discretion)
– Includes Business Attribute questions and Loyalty-based questions
• Typically have ratings of Satisfaction
• Transactional Satisfaction Surveys
– Focused on specific customer transaction
– Delivery is time-dependent on transaction occurrence
– Includes Business Attribute questions and Loyalty-based questions
• Typically have ratings of Satisfaction
• Targeted Surveys
– Focused on drilling into specific areas for deeper insight
– Delivery is on an as-needed basis
– Compliment the Relationship and Transactional surveys
• Three survey types can be used in any given Account
Company Surveys
Copyright © 2010 Business Over Broadway
• Customer surveys should result in reliable and valid
scores of customers’ attitudes
– Good measurement processes lead to good business decisions
• Reliability reflects the degree to which scores are
free from random error
– Reliability deals with precision/consistency
– Goal is to have a measurement system that
delivers reliable results
– Reliability varies from 0 (no reliability) to 1
(perfect reliability)
• Validity reflects the degree to which
scores are measuring what they
are designed to measure
– Validity deals with the meaning of the scores from the measurement system
– Goal is to have a measurement system that delivers valid results
Reliability and Validity
ValidNot Valid
ReliableNotReliable
Copyright © 2010 Business Over Broadway
• Four general forms of reliability
1. Inter-rater reliability1
– Degree of agreement between two or more
raters who are rating same attribute
2. Test-retest reliability1
– Degree of agreement between same
measure/rater over two time periods
3. Parallel-forms reliability
– Degree of agreement between two
measures (A vs. B) that measure same thing
4. Internal consistency reliability1
– Degree of consistency of ratings across
items within a measure
Reliability
=
Rater 1 Rater 2
=
Rater n Rater n
Attribute
1 Determination of reliability depends on number of questions in survey and types of surveys. Typically,
parallel-forms reliability cannot be studied as only one measure of customer satisfaction is available.
Time 1 Time 2
=
Rater n
Survey
Vers. A
Attribute
2
Attribute
m=
Survey
Vers. B=
Attribute
1
Rater n
Copyright © 2010 Business Over Broadway
• Three approaches at establishing validity
1. Content-related approach
– Examining the questions of the instrument to determine if they are a representative
sample of the universe of all possible questions
2. Criterion-related approach
– Correlating scores from the instrument with some other criteria
• Based on some theory of the customer satisfaction construct
• Customer satisfaction scores should be correlated to other criteria (e.g., other
measures of Account quality)
3. Construct-related approach
– Developing a nomological (lawful) network for the customer satisfaction scores
– Framework/Theory/Model for understanding and using customer satisfaction scores
should be supported by data
Validity
Copyright © 2010 Business Over Broadway
Evidence of Reliability
Copyright © 2010 Business Over Broadway
• Internal consistency determines extent to which
Contacts give similar ratings to items within a scale
– Indexed by three indices
1. Item-total correlations - correlation between rating of single item
to aggregate of remaining items in the survey
2. Split-half correlation – correlation between two halves of the
survey
3. Cronbach’s alpha – formula to calculate internal consistency
• Addresses the quality of the items in a scale
– If reliability is high, ratings do not tend to vary across items
– Expect some degree of internal consistency
• Do not expect perfect reliability – Perfect reliability would indicate
no variability in ratings and no need to ask multiple questions
Internal Consistency Reliability
Copyright © 2010 Business Over Broadway
Surveys
Contact 1
Contact 2
Contact n
Business
Attribute 1
Business
Attribute 2
Reliability Analysis
Internal Consistency Reliability
• Item-Total Correlation is the correlation of
specific attribute with aggregate of
remaining attributes
• Split-half reliability is the correlation
between average of the two halves of the
survey
• Cronbach’s alpha () estimate used to
determine internal consistency of ratings
(satisfaction) within a survey
• Calculated for any type of survey
that contains multiple questions
• Reliability varies from 0 (no internal
consistency) to 1 (perfect internal
consistency)
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
Business
Attribute 1
Business
Attribute 2
Business
Attribute m
.
.
.
=
Rater n
Attribute
2
Attribute
m=Attribute
1
Data Model for Internal Consistency
Reliability Analysis
Copyright © 2010 Business Over Broadway
Index (Measure) Definition Survey Questions
Customer Loyalty Index (4
questions)
Extent to which customers feel positively
toward COMPANY
Overall Satisfaction, Quality, Value, Recommend
Perceived Value of External
Information1 (7 questions)
Extent to which customers value external
sources of information regarding security
products
Security vendor websites, Whitepapers, Industry
analysts, Blogs / online communities, Security
publications, Peers, Webcasts / podcasts
Perceived Importance of
Doc/Mgmt/Reporting (3 questions)
Stated importance of Documentation,
Management, Reporting
Reporting, Management / Administration,
Documentation
Perceived Importance of Ease of
Use (3 questions)
Stated importance of user experience Ease of installation, End user experience, Ease of
upgrade
Quality of Security Product (8
questions)
Rating of quality of security product Security effectiveness, Reporting, Management /
Administration,
Documentation, Ease of installation, End user
experience, Ease of upgrade, Price
Quality of Technical Support (3
questions)
Rating of quality of technical support Technical support, Telephone, Email
Indices (Summated Scales)*
1 Items only appear in 2009 survey.
*Indices were created based on factor analyses of the survey questions; factor-analytic results show that specific survey items can be combined
together into one index. Survey questions that loaded highly on the same factor were combined into a single scale by averaging the responses
across items.
Factor analyses of survey items exposed several indices, each
measuring a general content area (e.g. loyalty, tech support).
Copyright © 2010 Business Over Broadway
Internal Consistency Reliability by
Decision Influence
Decision-Making Role
INDEX
Entire
Sample
Final
Decision Influencer Purchasing
2009 Indices
1 Customer Loyalty Index* .89 .89 .89 .88
2 Perceived Value of External
Information
.85 .85 .83 .87
3 Perceived Importance of
Doc/Mgmt/Reporting
.80 .78 .79 .78
4 Perceived Importance of Ease of Use .79 .76 .80 .80
5 Quality of Security Products .91 .92 .91 .92
6 Quality of Tech Support .91 .91 .90 .91
2010 Indices
1 Customer Loyalty Index .82 .81 .83 .81
2 Perceived Importance of
Doc/Mgmt/Reporting
.70 .69 .66 .78
3 Perceived Importance of Ease of Use .66 .68 .64 .58
4 Perceived Quality of Security Products .89 .89 .88 .91
5 Perceived Quality of Tech Support .92 .93 .92 .95
Internal Consistency Reliability1
Major Points
• Factor analyses of survey
items exposed several indices,
each measuring a general
content area (e.g. loyalty, tech
support).
• Definition of index is based on
the aggregated items
• Internal consistency
reliabilities are acceptable and
similar across all decision-
making categories.
• Each index would be useful for
executive dashboards since
each provides reliable, useful
information.
1 No significance testing possible for Cronbach’s alpha.
*See previous slide for description of each index and the survey items that represent each index.
Copyright © 2010 Business Over Broadway
• Inter-contact reliability determines extent to which
Contacts within an Account give similar customer
satisfaction survey ratings
– Indexed by a correlation between different Contacts within each Account across all
Accounts
• Inter-contact reliability addresses the consistency
of the implementation of the customer satisfaction
survey system across Contacts
– Expect some degree of inter-contact reliability because Contacts are rating the same
Account
• Do not want perfect reliability – Perfect reliability would indicate no utility of
surveying different management contacts within an organization
• Calculate inter-contact reliability for each survey
type across all business attributes
Inter-Rater Reliability
Copyright © 2010 Business Over Broadway
Data Model for Inter-Rater Reliability
Analysis
Type of Contact
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Inter-Contact Reliability
• Correlation of satisfaction ratings between
different Contacts within Accounts
• Calculated for each type of customer survey
• Reliability could vary from 0 (no agreement
between Contacts) to 1.0 (complete
agreement between Contacts)
.
.
.
...
=
Rater 2
Attribute
Rater 1
Copyright © 2010 Business Over Broadway
Inter-Contact Reliability (Decision-
makers vs. Influencers)
INDICES1
Decision
maker/
Influencer
1 Customer Loyalty Index* .19
2 Security Effectiveness -.12
3 Reporting .05
4 Management / Administration .15
5 Documentation -.04
6 Ease of Installation .16
7 End User Experience .02
8 Ease of Upgrade .22
9 Technical Support .05
10 Price .22
Major Points
• There is low agreement
between Contacts from different
decision-making standpoint.
• Contacts from different
decision-making standpoints
(e.g., Decision maker vs.
Influencer) show low agreement
in their ratings of COMPANY
• The degree of disagreement
across different survey
respondents within an Account
could reflect:
•Poor communication
within an Account
•Different expectations of
different functions
•Different usage of
COMPANY products
1 Survey data are from 2010 Survey; N = 47 to 50. No statistically significant correlations.
*Customer Loyalty Index is composite of following four questions: Overall Satisfaction, Quality, Value
and Recommend. Functional job function includes Finance,
Not enough sample to examine “Purchasing” role.
Copyright © 2010 Business Over Broadway
• Test-retest reliability determines extent to which
Contacts give similar ratings over time
– Indexed by a correlation between ratings across two time periods
– Higher correlation indicates that Contacts provide similar ratings over time
• Addresses the consistency of ratings over time
– If reliability is high, relative ordering of ratings do not tend to vary over time across
different Accounts
– Expect good degree of test-retest reliability because Contacts are rating the same
Account over time
• Contacts that are satisfied/dissatisfied at time 1 should also be
satisfied/dissatisfied at time 2
• Contacts with high/low value ratings at time 1 should also have high/low value
ratings at time 2
• Calculate test-retest reliability for satisfaction and
value ratings
Test-Retest Reliability
Copyright © 2010 Business Over Broadway
Data Model for Test-Retest Reliability
Analysis
Time 1
Survey
Time 2
Survey
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Account 1
Account 2
Account 3
Account m
Contact 1
Contact 2
Contact 3
Contact 4
Contact 5
Contact 6
Contact n-1
Contact n
Reliability Analysis
Test-Retest Reliability
• Correlation of satisfaction ratings between
same Contacts across two time periods
• Correlation of satisfaction ratings (averaged
over Contacts) between same Accounts
across two time periods
• Calculated only for Surveys that are
repeated
• Reliability varies from 0 (no
agreement between time 1 and time
2) to 1.0 (complete agreement
between time 1 and time 2)
.
.
.
.
.
.
.. .
=
Contact n Contact n
Time 1 Time 2
Copyright © 2010 Business Over Broadway
Test-Retest Reliability
INDICES/MEASURES
1 Customer Loyalty Index .69
2 Overall Satisfaction .32
3 Quality .58
4 Value .52
5 Recommend .70
6 Security Effectiveness .43
7 Reporting .53
8 Management / Administration .54
9 Documentation .44
10 Ease of Installation .50
11 End User Experience .49
12 Ease of Upgrade .45
13 Technical Support .51
14 Price .54
Test-Retest
Reliability1
1 Survey data are from 2009 (Time 1) and 2010 (Time 2). N = 472 to 478. All correlations significant at p < .01 level. CLI is a composite score
consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey
questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
=
Contact n Contact n
Time 1 Time 2
Major Points
• Test-retest reliability is high
• Customers tend to respond
in a similar way over two
time periods.
• Those who are satisfied
(dissatisfied) at time 1 tend
to be satisfied (dissatisfied)
at time 2.
Copyright © 2010 Business Over Broadway
Evidence of Validity
Copyright © 2010 Business Over Broadway
• Content validity is established when items in the
survey are a representative sample of some defined
universe of items
• Universe of survey items represents all items that
define the important elements of a company’s
performance
• Does the customer survey comprehensively assess
customer requirements?
• Must show “link” between set of items (survey) and
the universe of items
– Link determined by Subject Matter Experts (SMEs)
Content-related Approach
Copyright © 2010 Business Over Broadway
• Disloyal customers provide
comments regarding
improvements1
• Comments/issues range
from product to service issues
• All issues/reasons
covered by content of
customer survey
• Issues aligned with
results of driver analysis
Content-related Approach
1 Customer comments from those who rated Recommend question low.
Copyright © 2010 Business Over Broadway
• Criterion-related validity relies on statistical
analysis rather than judgment
• Established when scores on the survey are related
to some other variable (criterion) already known
to measure similar attributes
• Key is developing/selecting right criteria to use in
validation study
– Measures of Account Quality – by Account Team
– Objective Assessment Scores
Criterion-related Approach
Copyright © 2010 Business Over Broadway
• Construct-related validity addresses what is being
measured by examining relationships between
constructs (e.g., customer satisfaction, customer
loyalty, other measures of Account quality)
• Concerned with theory and the proposed relationships
between constructs based on that theory
• Understanding the theoretical relationship of
constructs to other constructs
– To what should customer satisfaction be (convergent validity)
and not be related (discriminant validity)?
• Need to examine patterns of relationships using the
survey and other instruments
Construct-related Approach
Copyright © 2010 Business Over Broadway
2009 Mean SD N 1 2 3 4 5 6
1. Customer Loyalty Index 4.84 0.94 2941 0.89
2. Quality of Security Product 4.59 0.86 2917 0.77 0.91
3. Quality of Technical Support 4.81 1.09 2880 0.68 0.69 0.91
4. Perceived Value of External Information 4.34 0.97 2787 0.31 0.37 0.29 0.85
5. Perceived Importance of Doc/Mgmt/Reporting 4.77 0.87 2914 0.29 0.43 0.31 0.34 0.80
6. Perceived Importance of Ease of Use 5.05 0.87 2919 0.33 0.48 0.33 0.28 0.54 0.79
Correlations Among Indices
 Indices are correlated logically with each other
 Higher ratings on key indices (quality, value) correspond to higher
customer loyalty ratings compared to other indices (perceived
importance)
2010 Mean SD N 1 2 3 4 5
1. Customer Loyalty Index 5.54 1.27 3449 0.82
2. Quality of Security Product 4.72 0.75 3417 0.78 0.89
3. Quality of Technical Support 4.95 1.04 3387 0.67 0.65 0.92
4. Perceived Importance of Doc/Mgmt/Reporting 5.01 0.66 3434 0.11 0.21 0.12 0.70
5. Perceived Importance of Ease of Use 5.17 0.69 3437 0.10 0.23 0.12 0.41 0.66
1 Cronbach’s alpha (internal consistency estimates) are located in the diagonal.
Copyright © 2010 Business Over Broadway
• Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2009)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 4.84 2941
Understands Needs 4.57 2926 0.73
Addresses Needs 4.52 2924 0.78
Understands Better Than Competitors 4.30 2884 0.69
Security Effectiveness 5.18 2911 0.70
Reporting 4.40 2875 0.54
Management / Administration 4.52 2866 0.63
Documentation 4.23 2871 0.55
Ease of Installation 4.61 2884 0.58
End User Experience 4.66 2880 0.66
Ease of Upgrade 4.77 2856 0.61
Technical Support 4.80 2860 0.67
Price 4.35 2877 0.55
Compare COMPANY to Alternatives 4.06 2645 0.69
Tech Support via Telephone 4.96 1815 0.61
Tech Support via Email 4.79 1792 0.62
Tech Support via Web Knowledgebase 4.39 2102 0.49
1 No significance testing possible for Cronbach’s alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
Copyright © 2010 Business Over Broadway
• Driver analysis shows that customers who are more
satisfied also report higher levels of customer loyalty
Construct-related Approach (2010)
Mean
Rating N
Impact on
Loyalty
Customer Loyalty Index 5.54 3449
Understands Needs 4.32 3414 0.44
Addresses Needs 4.31 3408 0.48
Understands Better Than Competitors 4.37 3337 0.52
Security Effectiveness 5.21 3407 0.65
Reporting 4.58 3371 0.54
Management / Administration 4.67 3387 0.67
Documentation 4.48 3372 0.54
Ease of Installation 4.72 3382 0.56
End User Experience 4.73 3371 0.66
Ease of Upgrade 4.78 3356 0.59
Technical Support 4.91 3360 0.66
Price 4.62 3348 0.47
Compare COMPANY to Alternatives 4.01 3020 0.77
Tech Support via Telephone 5.05 2408 0.63
Tech Support via Email 4.91 2337 0.64
Tech Support via Web Knowledgebase 4.47 2585 0.55
1 No significance testing possible for Cronbach’s alpha.
*CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were
selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they
measure the same construct.
Copyright © 2010 Business Over Broadway
• Use previous validity approaches (content and
criterion) to support a theoretical model of customer
satisfaction
• Various theoretical models
state that customer loyalty
is impacted by satisfaction with
product and satisfaction with
service
• Need to examine the pattern of
relationships among these
constructs
• Pattern of correlations should support this model
Construct-related Approach
Service
Satisfaction
Product
Satisfaction
Customer
Loyalty
Basic Theory of
Customer Satisfaction
Copyright © 2010 Business Over Broadway
• Test the model by examining
correlations among three
variables:
– Product Quality (Quality of Security Eff.)
– Tech Support Quality
– Customer Loyalty (CLI)
• Path analysis1 applied to
observed correlations
– Isolates degree of impact a given
variable has on other variables
• Analysis supports theory that
both product quality and tech
tech support quality impact
customer loyalty independently
Construct-related Approach
CORRELATIONS
CLI
Tech
Qual
Product Quality .70 .53
Customer Loyalty (CLI) - .68
1 For detailed description of path analysis methodology, please see: Pedhazur, E. J. (1982). Multiple
regression in behavioral research: Explanation and prediction (2nd Ed.) Holt, Rinehart and Winston.
New York, NY.
Tech Support
Product
Effectiveness
Customer
Loyalty
Path Analysis
.11
.54
.51
Copyright © 2010 Business Over Broadway
• Support Level and Customer Loyalty
• Three levels of Support (Standard, Premium, Platinum)
• Lower customer loyalty reported by customers with
Platinum support
– Platinum support customers could have higher expectations than
Standard support customers.
Support Level
Copyright © 2010 Business Over Broadway
• Difference found across regions1
– APAC has highest levels of customer loyalty and quality ratings
– Southern Europe as lowest levels of customer loyalty and quality
ratings
Segmentation By Region
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
Segmentation By Industry
Difference found across industries1
HED has the lowest levels of customer loyalty
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
Segmentation By License Tier
Difference found across license tiers1
License tiers 3 and 4 have highest levels of customer loyalty and quality ratings
1 Based on survey results from 2010.
Copyright © 2010 Business Over Broadway
• Internal consistency reliability
– Contacts tend to give similar ratings across the different items of the same
scale
• Non-perfect reliability indicates that different items still provide different
information about customers’ attitudes
• Reasons for extremely high reliability
– High degree of redundancy of items
– Rater fatigue due to survey length so respondents rate all questions similarly
• Inter-contact reliability
– Contacts within Accounts show little agreement in ratings
– Different types of Contacts provide different vantage point when rating the
company
• Poor communication within Account across levels and/job functions
• Different Contacts have differing expectations
• Contacts are treated differently by Account team depending on their
level/function
Summary of Reliability of Survey Ratings
Copyright © 2010 Business Over Broadway
• Content-related approach
– Questions in customer survey are representative sample
of universe of questions
• Items comprehensively cover customer requirements
throughout all phases of the customer lifecycle
• Dissatisfied customers’ open-ended comments are addressed
by questions in the survey
• Criterion-related approach
– No external criteria are available at this time
• Construct-related approach
– Pattern of relationships of customer satisfaction data
supports theory of customer satisfaction
• Customer satisfaction is related to customer loyalty
• Product quality impacts loyalty through perceived value
Summary of Validity
Copyright © 2010 Business Over Broadway
• Self-assessment survey identified strengths and areas
needing improvement1
• Overall, low to moderate adoption rates of best practices
across all components of customer feedback programs
– Customer feedback not as important as financial metrics,
customer feedback not integrated with CRM system, problem
resolution not integrated into CRM system
• Strengths seen in specific areas of COMPANY’S customer
feedback program:
– Top executive is champion of the program, multiple methods of
data collection, Web surveys, various measures of customer
loyalty, customer research presented externally, existing
customer data used for segmentation
Customer Feedback Programs Best
Practices Survey
1 Survey identifies extent to which company adopts best practices in their customer feedback program. EMPLOYEE JANE DOE
completed the online survey (http://www.businessoverbroadway.com/cfpbpi.htm) for COMPANY on June 10, 2010. See survey for
questions.
Copyright © 2010 Business Over Broadway
• The company’s survey process produces reliable,
valid indices of customers’ attitudes
• Evidence of reliability
– All indices are highly reliable – items within indices
can be combined to create summary scores
• Evidence of validity
– Survey questions comprehensively assess important
areas to the customers
– Each index measures unique construct apart from the
other indices
– Relationships among indices supports customer
satisfaction models
Conclusions
Copyright © 2010 Business Over Broadway
• The company can feel confident using customer
survey results when making business decisions
– Identifying drivers of customer loyalty
– Using Indexes to manage customer relationships
• Understand why different customers within the
same Account hold different attitudes toward
COMPANY
– Do they communicate with each other (through
Company-sanctioned User groups or individually)? They
could get together and talk about the benefits of
COMPANY. Could COMPANY assist in this process?
Conclusions
Copyright © 2010 Business Over Broadway
• Changes in Customer Loyalty Measurement
– Put loyalty question at start of survey (to assess top of mind
responses) with common rating scale (recommend 0 to 10)
scale)
– Add questions to measure other components of customer
loyalty
• Purchasing Loyalty: Example: How likely are you to purchase
additional/different types of products from COMPANY?
• Retention Loyalty: Example: How likely are you to switch to a different
security provider within the next 12 months?
• Benefits
– Make decisions to grow business through new and existing
customers
– Loyalty indices can be used to build customer-centricity
• Executive dashboards, guide strategy/governance, resource
allocation, incentive compensation, benchmark against industry
standards
Recommendations on Method
Copyright © 2010 Business Over Broadway
• Changes in Technical Support Measurement
– Move all technical support questions into one section
• Telephone, Email, and technical support
– Consider adding other technical support questions to
broaden technical support coverage
• Knowledge, professionalism, care
• Benefits
– Easier for customers to complete survey
– Provide more specific feedback to technical support
personnel
• Consider ways to increase response rates
– Email invitation content
Recommendations on Method
Copyright © 2010 Business Over Broadway
• Integrate customer feedback into company’s
strategy/governance1
– Company’s strategic vision, mission and goals; executives’
objectives and incentive compensation, front-line employees’
objectives and incentive compensation
• Integrate customer feedback into daily business
processes1
– CRM system; company/executive dashboards; communicate
results regularly to the entire company
• Conduct in-depth research using customer survey results1
– Linkage analysis between customer feedback and operational
metrics, financial metrics, employee metrics.
Recommendations on Strategy,
Integration and Research
1 Based on Customer Feedback Programs Self-Assessment Survey Results.
Copyright © 2010 Business Over Broadway
• Consider other “financial” metrics to track and link to
customer survey
– Consult with CFO to consider other financial metrics that are
tracked and consider using those to identify which aspects of
customer satisfaction/loyalty relate to financial growth of
COMPANY
• e.g. Renewals
– Track financial metrics over time in conjunction with the
customer survey to conduct longitudinal studies
• Study impact of attitudes at time 1 on financial metrics at time 2 and
beyond
• Linkage study between COMPANY’S transactional survey
and relationship survey
– To what extent do transactions (e.g. in call center) impact long-
term customer loyalty
Recommendations for Future Research
Copyright © 2010 Business Over Broadway
For More Information
Business growth through customer insight
Business Over Broadway
Bob E. Hayes, Ph.D.
Email: bob@businessoverbroadway.com
Web: www.businessoverbroadway.com
Blog: www.businessoverbroadway.blogspot.com
Twitter: www.twitter.com/bobehayes

More Related Content

What's hot

Assistant bank manager performance appraisal
Assistant bank manager performance appraisalAssistant bank manager performance appraisal
Assistant bank manager performance appraisalTeddySheringham012
 
Information systems audit and control
Information systems audit and controlInformation systems audit and control
Information systems audit and controlKashif Rana ACCA
 
Office administration assistant performance appraisal
Office administration assistant performance appraisalOffice administration assistant performance appraisal
Office administration assistant performance appraisal
calebmoore856
 
Introductory session on business analyst training1
Introductory session on business analyst training1Introductory session on business analyst training1
Introductory session on business analyst training1Suprriya Nair
 
It enterprise architect performance appraisal
It enterprise architect performance appraisalIt enterprise architect performance appraisal
It enterprise architect performance appraisal
morganabbie765
 
How to define business problem statement
How to define business problem statementHow to define business problem statement
How to define business problem statement
Santosh Kumar Singh
 
Technical support engineer performance appraisal
Technical support engineer performance appraisalTechnical support engineer performance appraisal
Technical support engineer performance appraisal
alexanderhill006
 
Training manager performance appraisal
Training manager performance appraisalTraining manager performance appraisal
Training manager performance appraisal
martinezrosie780
 
Medical coding specialist performance appraisal
Medical coding specialist performance appraisalMedical coding specialist performance appraisal
Medical coding specialist performance appraisalJamieCarragher345
 
6 benefits of internal auditing
6 benefits of internal auditing6 benefits of internal auditing
6 benefits of internal auditing
SALIH AHMED ISLAM
 
Concepts Of business analyst Practices - Part 1
Concepts Of business analyst Practices - Part 1Concepts Of business analyst Practices - Part 1
Concepts Of business analyst Practices - Part 1
Moutasm Tamimi
 
Purchasing specialist performance appraisal
Purchasing specialist performance appraisalPurchasing specialist performance appraisal
Purchasing specialist performance appraisal
howardmathew11
 
Pmo administrator performance appraisal
Pmo administrator performance appraisalPmo administrator performance appraisal
Pmo administrator performance appraisal
emilymichelle224
 
Medical billing specialist performance appraisal
Medical billing specialist performance appraisalMedical billing specialist performance appraisal
Medical billing specialist performance appraisal
michealtaylor431
 
Employee Performance Review
Employee Performance ReviewEmployee Performance Review
Employee Performance ReviewMindy Bates
 
Senior network engineer performance appraisal
Senior network engineer performance appraisalSenior network engineer performance appraisal
Senior network engineer performance appraisal
jamespoter576
 
Bank branch manager performance appraisal
Bank branch manager performance appraisalBank branch manager performance appraisal
Bank branch manager performance appraisal
martinjack417
 
Training coordinator performance appraisal
Training coordinator performance appraisalTraining coordinator performance appraisal
Training coordinator performance appraisal
martinezrosie780
 
Chapter 10 Survey Research Communicating with Respondents .pptx
Chapter 10 Survey Research Communicating with Respondents .pptxChapter 10 Survey Research Communicating with Respondents .pptx
Chapter 10 Survey Research Communicating with Respondents .pptx
RajjaRashad1
 
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdfIESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
Sazzad Hossain, ITP, MBA, CSCA™
 

What's hot (20)

Assistant bank manager performance appraisal
Assistant bank manager performance appraisalAssistant bank manager performance appraisal
Assistant bank manager performance appraisal
 
Information systems audit and control
Information systems audit and controlInformation systems audit and control
Information systems audit and control
 
Office administration assistant performance appraisal
Office administration assistant performance appraisalOffice administration assistant performance appraisal
Office administration assistant performance appraisal
 
Introductory session on business analyst training1
Introductory session on business analyst training1Introductory session on business analyst training1
Introductory session on business analyst training1
 
It enterprise architect performance appraisal
It enterprise architect performance appraisalIt enterprise architect performance appraisal
It enterprise architect performance appraisal
 
How to define business problem statement
How to define business problem statementHow to define business problem statement
How to define business problem statement
 
Technical support engineer performance appraisal
Technical support engineer performance appraisalTechnical support engineer performance appraisal
Technical support engineer performance appraisal
 
Training manager performance appraisal
Training manager performance appraisalTraining manager performance appraisal
Training manager performance appraisal
 
Medical coding specialist performance appraisal
Medical coding specialist performance appraisalMedical coding specialist performance appraisal
Medical coding specialist performance appraisal
 
6 benefits of internal auditing
6 benefits of internal auditing6 benefits of internal auditing
6 benefits of internal auditing
 
Concepts Of business analyst Practices - Part 1
Concepts Of business analyst Practices - Part 1Concepts Of business analyst Practices - Part 1
Concepts Of business analyst Practices - Part 1
 
Purchasing specialist performance appraisal
Purchasing specialist performance appraisalPurchasing specialist performance appraisal
Purchasing specialist performance appraisal
 
Pmo administrator performance appraisal
Pmo administrator performance appraisalPmo administrator performance appraisal
Pmo administrator performance appraisal
 
Medical billing specialist performance appraisal
Medical billing specialist performance appraisalMedical billing specialist performance appraisal
Medical billing specialist performance appraisal
 
Employee Performance Review
Employee Performance ReviewEmployee Performance Review
Employee Performance Review
 
Senior network engineer performance appraisal
Senior network engineer performance appraisalSenior network engineer performance appraisal
Senior network engineer performance appraisal
 
Bank branch manager performance appraisal
Bank branch manager performance appraisalBank branch manager performance appraisal
Bank branch manager performance appraisal
 
Training coordinator performance appraisal
Training coordinator performance appraisalTraining coordinator performance appraisal
Training coordinator performance appraisal
 
Chapter 10 Survey Research Communicating with Respondents .pptx
Chapter 10 Survey Research Communicating with Respondents .pptxChapter 10 Survey Research Communicating with Respondents .pptx
Chapter 10 Survey Research Communicating with Respondents .pptx
 
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdfIESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
IESBA-Code-of-Ethics-and-International-Independence-Standards-1.pdf
 

Viewers also liked

Value proposition &amp; usp
Value proposition &amp; uspValue proposition &amp; usp
Value proposition &amp; usp
Vikash Kumar Bibhakar
 
Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011Mikael Sandblom
 
Customer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTDCustomer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTD
Mahbubun Nabi Rasel
 
Textbook Campaign
Textbook CampaignTextbook Campaign
Textbook Campaign
MSCSA
 
Verizon Presentation
Verizon PresentationVerizon Presentation
Verizon PresentationLiang Huang
 
Columbus Metropolitan Library Survey
Columbus Metropolitan Library SurveyColumbus Metropolitan Library Survey
Columbus Metropolitan Library Survey
Julie Theado
 

Viewers also liked (8)

Value proposition &amp; usp
Value proposition &amp; uspValue proposition &amp; usp
Value proposition &amp; usp
 
Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011Customer Satisfaction Survey Results 2011
Customer Satisfaction Survey Results 2011
 
updated CV Yacoub
updated CV Yacoubupdated CV Yacoub
updated CV Yacoub
 
Customer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTDCustomer Satisfaction Survey On Robi Axiata LTD
Customer Satisfaction Survey On Robi Axiata LTD
 
Textbook Campaign
Textbook CampaignTextbook Campaign
Textbook Campaign
 
Verizon Presentation
Verizon PresentationVerizon Presentation
Verizon Presentation
 
Columbus Metropolitan Library Survey
Columbus Metropolitan Library SurveyColumbus Metropolitan Library Survey
Columbus Metropolitan Library Survey
 
Usp Marketing
Usp MarketingUsp Marketing
Usp Marketing
 

Similar to Validation of Customer Survey

Developing a Customer Centric Research Program
Developing a Customer Centric Research ProgramDeveloping a Customer Centric Research Program
Developing a Customer Centric Research ProgramBusiness Over Broadway
 
Customer Service Excellence
Customer Service ExcellenceCustomer Service Excellence
Customer Service Excellence
Kathryn Larkin
 
A Robust approach to analysis of Customer Satisfaction Survey feedback
A  Robust approach to analysis of Customer Satisfaction Survey feedbackA  Robust approach to analysis of Customer Satisfaction Survey feedback
A Robust approach to analysis of Customer Satisfaction Survey feedback
Santanoo Medhi
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
Vishal Kumar
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
TCELab LLC
 
Big Data and Customer Experience
Big Data and Customer ExperienceBig Data and Customer Experience
Big Data and Customer Experience
Business Over Broadway
 
Customer Satisfaction in Shared Services
Customer Satisfaction in Shared ServicesCustomer Satisfaction in Shared Services
Customer Satisfaction in Shared Services
ScottMadden, Inc.
 
Quality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin ShekapureQuality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin Shekapure
Nitin Shekapure
 
Building a Customer Feedback Program
Building a Customer Feedback ProgramBuilding a Customer Feedback Program
Building a Customer Feedback Program
Business Over Broadway
 
Quality control
Quality controlQuality control
Quality control
Nitin Shekapure
 
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01TCELab LLC
 
A Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation ProcessA Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation Process
Tracy Drey
 
Improving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingImproving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingNuwan Ireshinie
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellence
kushshah
 
Service quality
Service qualityService quality
Service quality
ADEEGGA AD AGENCY
 
Customer Experience Management
Customer Experience ManagementCustomer Experience Management
Customer Experience Management
BRIDGEi2i Analytics Solutions
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
TCELab LLC
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
Vishal Kumar
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
TCELab LLC
 

Similar to Validation of Customer Survey (20)

Developing a Customer Centric Research Program
Developing a Customer Centric Research ProgramDeveloping a Customer Centric Research Program
Developing a Customer Centric Research Program
 
Customer Service Excellence
Customer Service ExcellenceCustomer Service Excellence
Customer Service Excellence
 
A Robust approach to analysis of Customer Satisfaction Survey feedback
A  Robust approach to analysis of Customer Satisfaction Survey feedbackA  Robust approach to analysis of Customer Satisfaction Survey feedback
A Robust approach to analysis of Customer Satisfaction Survey feedback
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Big Data and Customer Experience
Big Data and Customer ExperienceBig Data and Customer Experience
Big Data and Customer Experience
 
Customer Satisfaction in Shared Services
Customer Satisfaction in Shared ServicesCustomer Satisfaction in Shared Services
Customer Satisfaction in Shared Services
 
Quality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin ShekapureQuality Control Importance and Tools use for QC by Nitin Shekapure
Quality Control Importance and Tools use for QC by Nitin Shekapure
 
dasat
dasatdasat
dasat
 
Building a Customer Feedback Program
Building a Customer Feedback ProgramBuilding a Customer Feedback Program
Building a Customer Feedback Program
 
Quality control
Quality controlQuality control
Quality control
 
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
Bobhayestcebigdatawebinar03272013 130417142258-phpapp01
 
A Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation ProcessA Research Paper On Customer Satisfaction Evaluation Process
A Research Paper On Customer Satisfaction Evaluation Process
 
Improving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service MarketingImproving Service Quality and Productivity - Service Marketing
Improving Service Quality and Productivity - Service Marketing
 
Game Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational ExcellenceGame Changing Quality Strategies that Drive Organizational Excellence
Game Changing Quality Strategies that Drive Organizational Excellence
 
Service quality
Service qualityService quality
Service quality
 
Customer Experience Management
Customer Experience ManagementCustomer Experience Management
Customer Experience Management
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
 

More from Business Over Broadway

What MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysWhat MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and Surveys
Business Over Broadway
 
Investigating data scientists
Investigating data scientistsInvestigating data scientists
Investigating data scientists
Business Over Broadway
 
In a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment IndexIn a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment Index
Business Over Broadway
 
The Hidden Bias in Customer Metrics
The Hidden Bias in Customer MetricsThe Hidden Bias in Customer Metrics
The Hidden Bias in Customer Metrics
Business Over Broadway
 
Big Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience ProfessionalsBig Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience Professionals
Business Over Broadway
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
Business Over Broadway
 
Customer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample ReportCustomer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample Report
Business Over Broadway
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
Business Over Broadway
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
Business Over Broadway
 
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship SurveyAsking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
Business Over Broadway
 
Linkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback ProgramsLinkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback Programs
Business Over Broadway
 
Competitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer LoyaltyCompetitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer Loyalty
Business Over Broadway
 
Managing Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachManaging Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachBusiness Over Broadway
 
RAPID Loyalty Measurement
RAPID Loyalty MeasurementRAPID Loyalty Measurement
RAPID Loyalty Measurement
Business Over Broadway
 

More from Business Over Broadway (14)

What MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and SurveysWhat MBA Students Need to Know about CX, Data Science and Surveys
What MBA Students Need to Know about CX, Data Science and Surveys
 
Investigating data scientists
Investigating data scientistsInvestigating data scientists
Investigating data scientists
 
In a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment IndexIn a Word: The Customer Sentiment Index
In a Word: The Customer Sentiment Index
 
The Hidden Bias in Customer Metrics
The Hidden Bias in Customer MetricsThe Hidden Bias in Customer Metrics
The Hidden Bias in Customer Metrics
 
Big Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience ProfessionalsBig Data - What it Really Means for VOC and Customer Experience Professionals
Big Data - What it Really Means for VOC and Customer Experience Professionals
 
Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...Improving the customer experience using big data customer-centric measurement...
Improving the customer experience using big data customer-centric measurement...
 
Customer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample ReportCustomer Relationship Diagnostic: Sample Report
Customer Relationship Diagnostic: Sample Report
 
Customer Experience Management for Startups
Customer Experience Management for StartupsCustomer Experience Management for Startups
Customer Experience Management for Startups
 
Big Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience ManagementBig Data has Big Implications for Customer Experience Management
Big Data has Big Implications for Customer Experience Management
 
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship SurveyAsking the Right CX Questions: Optimizing your Customer Relationship Survey
Asking the Right CX Questions: Optimizing your Customer Relationship Survey
 
Linkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback ProgramsLinkage Analysis in Customer Feedback Programs
Linkage Analysis in Customer Feedback Programs
 
Competitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer LoyaltyCompetitive Analytics that Drive Customer Loyalty
Competitive Analytics that Drive Customer Loyalty
 
Managing Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro ApproachManaging Customer Loyalty - Micro and Macro Approach
Managing Customer Loyalty - Micro and Macro Approach
 
RAPID Loyalty Measurement
RAPID Loyalty MeasurementRAPID Loyalty Measurement
RAPID Loyalty Measurement
 

Recently uploaded

Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
creerey
 
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
bosssp10
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Arihant Webtech Pvt. Ltd
 
Project File Report BBA 6th semester.pdf
Project File Report BBA 6th semester.pdfProject File Report BBA 6th semester.pdf
Project File Report BBA 6th semester.pdf
RajPriye
 
Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...
dylandmeas
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
agatadrynko
 
Brand Analysis for an artist named Struan
Brand Analysis for an artist named StruanBrand Analysis for an artist named Struan
Brand Analysis for an artist named Struan
sarahvanessa51503
 
Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111
zoyaansari11365
 
LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024
Lital Barkan
 
Authentically Social Presented by Corey Perlman
Authentically Social Presented by Corey PerlmanAuthentically Social Presented by Corey Perlman
Authentically Social Presented by Corey Perlman
Corey Perlman, Social Media Speaker and Consultant
 
amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05
marketing317746
 
Recruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media MasterclassRecruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media Masterclass
LuanWise
 
Premium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern BusinessesPremium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern Businesses
SynapseIndia
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
Nicola Wreford-Howard
 
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
Lviv Startup Club
 
Building Your Employer Brand with Social Media
Building Your Employer Brand with Social MediaBuilding Your Employer Brand with Social Media
Building Your Employer Brand with Social Media
LuanWise
 
Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...
Lviv Startup Club
 
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
Boris Ziegler
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
uae taxgpt
 
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challengesEvent Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
Holger Mueller
 

Recently uploaded (20)

Cree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBdCree_Rey_BrandIdentityKit.PDF_PersonalBd
Cree_Rey_BrandIdentityKit.PDF_PersonalBd
 
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
Call 8867766396 Satta Matka Dpboss Matka Guessing Satta batta Matka 420 Satta...
 
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdfSearch Disrupted Google’s Leaked Documents Rock the SEO World.pdf
Search Disrupted Google’s Leaked Documents Rock the SEO World.pdf
 
Project File Report BBA 6th semester.pdf
Project File Report BBA 6th semester.pdfProject File Report BBA 6th semester.pdf
Project File Report BBA 6th semester.pdf
 
Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...Discover the innovative and creative projects that highlight my journey throu...
Discover the innovative and creative projects that highlight my journey throu...
 
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdfikea_woodgreen_petscharity_dog-alogue_digital.pdf
ikea_woodgreen_petscharity_dog-alogue_digital.pdf
 
Brand Analysis for an artist named Struan
Brand Analysis for an artist named StruanBrand Analysis for an artist named Struan
Brand Analysis for an artist named Struan
 
Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111Introduction to Amazon company 111111111111
Introduction to Amazon company 111111111111
 
LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024LA HUG - Video Testimonials with Chynna Morgan - June 2024
LA HUG - Video Testimonials with Chynna Morgan - June 2024
 
Authentically Social Presented by Corey Perlman
Authentically Social Presented by Corey PerlmanAuthentically Social Presented by Corey Perlman
Authentically Social Presented by Corey Perlman
 
amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05amptalk_RecruitingDeck_english_2024.06.05
amptalk_RecruitingDeck_english_2024.06.05
 
Recruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media MasterclassRecruiting in the Digital Age: A Social Media Masterclass
Recruiting in the Digital Age: A Social Media Masterclass
 
Premium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern BusinessesPremium MEAN Stack Development Solutions for Modern Businesses
Premium MEAN Stack Development Solutions for Modern Businesses
 
Exploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social DreamingExploring Patterns of Connection with Social Dreaming
Exploring Patterns of Connection with Social Dreaming
 
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
Evgen Osmak: Methods of key project parameters estimation: from the shaman-in...
 
Building Your Employer Brand with Social Media
Building Your Employer Brand with Social MediaBuilding Your Employer Brand with Social Media
Building Your Employer Brand with Social Media
 
Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...Kseniya Leshchenko: Shared development support service model as the way to ma...
Kseniya Leshchenko: Shared development support service model as the way to ma...
 
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
Agency Managed Advisory Board As a Solution To Career Path Defining Business ...
 
VAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and RequirementsVAT Registration Outlined In UAE: Benefits and Requirements
VAT Registration Outlined In UAE: Benefits and Requirements
 
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challengesEvent Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
Event Report - SAP Sapphire 2024 Orlando - lots of innovation and old challenges
 

Validation of Customer Survey

  • 1. Copyright © 2010 Business Over Broadway Validation of COMPANY Customer Satisfaction Survey Business growth through customer insight Business Over Broadway
  • 2. Copyright © 2010 Business Over Broadway • Executive Summary • Components of Customer Feedback Programs • Uses of Customer Feedback Data • Company Surveys • Definitions of Reliability and Validity • Evidence of Reliability • Evidence of Validity • Summary • Conclusions • Recommendations Overview
  • 3. Copyright © 2010 Business Over Broadway • The company can feel confident using their customer survey results when making business decisions • Loyalty questions can be used to make reliable, useful decisions to grow business through new and existing customers • Build customer-centricity in COMPANY by using loyalty indices in: – executive dashboards, strategy/governance, decision making, incentive compensation, benchmark against industry standards • Proposed Recommendations – Change Customer Loyalty Measurement – Change Technical Support Measurement – Adopt best practices in Strategy/Governance, Business Integration Executive Summary
  • 4. Copyright © 2010 Business Over Broadway • Adoption of best practices in each area ensures program success1 Components of Customer Feedback Programs 1 In Hayes, B.E. (2009). Beyond the Ultimate Question: A Systematic Approach to Improve Customer Loyalty. Quality Press. Milwaukee, WI. Loyalty Leading companies adopt specific business practices compared to Loyalty Lagging companies
  • 5. Copyright © 2010 Business Over Broadway Employee, Partner and Customer Loyalty Drive Business Results Based on the book, The Service Profit Chain: How leading Companies Link Profit and Growth to Loyalty, Satisfaction and Value (Heskett, Sasser & Schlesinger, 1997) Employee Survey Customer Survey Partner Survey Business Strategy Business Programs Company PerformanceInternal System Employee Satisfaction Employee Productivity Perceived Value Customer Satisfaction Revenue Profits Employee Loyalty Customer Loyalty Partner Advocacy Purchasing Retention Market Share
  • 6. Copyright © 2010 Business Over Broadway • Relationship Satisfaction Surveys – Focused on overall relationship with the customer – Delivery is not time-dependent (done at company discretion) – Includes Business Attribute questions and Loyalty-based questions • Typically have ratings of Satisfaction • Transactional Satisfaction Surveys – Focused on specific customer transaction – Delivery is time-dependent on transaction occurrence – Includes Business Attribute questions and Loyalty-based questions • Typically have ratings of Satisfaction • Targeted Surveys – Focused on drilling into specific areas for deeper insight – Delivery is on an as-needed basis – Compliment the Relationship and Transactional surveys • Three survey types can be used in any given Account Company Surveys
  • 7. Copyright © 2010 Business Over Broadway • Customer surveys should result in reliable and valid scores of customers’ attitudes – Good measurement processes lead to good business decisions • Reliability reflects the degree to which scores are free from random error – Reliability deals with precision/consistency – Goal is to have a measurement system that delivers reliable results – Reliability varies from 0 (no reliability) to 1 (perfect reliability) • Validity reflects the degree to which scores are measuring what they are designed to measure – Validity deals with the meaning of the scores from the measurement system – Goal is to have a measurement system that delivers valid results Reliability and Validity ValidNot Valid ReliableNotReliable
  • 8. Copyright © 2010 Business Over Broadway • Four general forms of reliability 1. Inter-rater reliability1 – Degree of agreement between two or more raters who are rating same attribute 2. Test-retest reliability1 – Degree of agreement between same measure/rater over two time periods 3. Parallel-forms reliability – Degree of agreement between two measures (A vs. B) that measure same thing 4. Internal consistency reliability1 – Degree of consistency of ratings across items within a measure Reliability = Rater 1 Rater 2 = Rater n Rater n Attribute 1 Determination of reliability depends on number of questions in survey and types of surveys. Typically, parallel-forms reliability cannot be studied as only one measure of customer satisfaction is available. Time 1 Time 2 = Rater n Survey Vers. A Attribute 2 Attribute m= Survey Vers. B= Attribute 1 Rater n
  • 9. Copyright © 2010 Business Over Broadway • Three approaches at establishing validity 1. Content-related approach – Examining the questions of the instrument to determine if they are a representative sample of the universe of all possible questions 2. Criterion-related approach – Correlating scores from the instrument with some other criteria • Based on some theory of the customer satisfaction construct • Customer satisfaction scores should be correlated to other criteria (e.g., other measures of Account quality) 3. Construct-related approach – Developing a nomological (lawful) network for the customer satisfaction scores – Framework/Theory/Model for understanding and using customer satisfaction scores should be supported by data Validity
  • 10. Copyright © 2010 Business Over Broadway Evidence of Reliability
  • 11. Copyright © 2010 Business Over Broadway • Internal consistency determines extent to which Contacts give similar ratings to items within a scale – Indexed by three indices 1. Item-total correlations - correlation between rating of single item to aggregate of remaining items in the survey 2. Split-half correlation – correlation between two halves of the survey 3. Cronbach’s alpha – formula to calculate internal consistency • Addresses the quality of the items in a scale – If reliability is high, ratings do not tend to vary across items – Expect some degree of internal consistency • Do not expect perfect reliability – Perfect reliability would indicate no variability in ratings and no need to ask multiple questions Internal Consistency Reliability
  • 12. Copyright © 2010 Business Over Broadway Surveys Contact 1 Contact 2 Contact n Business Attribute 1 Business Attribute 2 Reliability Analysis Internal Consistency Reliability • Item-Total Correlation is the correlation of specific attribute with aggregate of remaining attributes • Split-half reliability is the correlation between average of the two halves of the survey • Cronbach’s alpha () estimate used to determine internal consistency of ratings (satisfaction) within a survey • Calculated for any type of survey that contains multiple questions • Reliability varies from 0 (no internal consistency) to 1 (perfect internal consistency) Business Attribute m Business Attribute 1 Business Attribute 2 Business Attribute m Business Attribute 1 Business Attribute 2 Business Attribute m . . . = Rater n Attribute 2 Attribute m=Attribute 1 Data Model for Internal Consistency Reliability Analysis
  • 13. Copyright © 2010 Business Over Broadway Index (Measure) Definition Survey Questions Customer Loyalty Index (4 questions) Extent to which customers feel positively toward COMPANY Overall Satisfaction, Quality, Value, Recommend Perceived Value of External Information1 (7 questions) Extent to which customers value external sources of information regarding security products Security vendor websites, Whitepapers, Industry analysts, Blogs / online communities, Security publications, Peers, Webcasts / podcasts Perceived Importance of Doc/Mgmt/Reporting (3 questions) Stated importance of Documentation, Management, Reporting Reporting, Management / Administration, Documentation Perceived Importance of Ease of Use (3 questions) Stated importance of user experience Ease of installation, End user experience, Ease of upgrade Quality of Security Product (8 questions) Rating of quality of security product Security effectiveness, Reporting, Management / Administration, Documentation, Ease of installation, End user experience, Ease of upgrade, Price Quality of Technical Support (3 questions) Rating of quality of technical support Technical support, Telephone, Email Indices (Summated Scales)* 1 Items only appear in 2009 survey. *Indices were created based on factor analyses of the survey questions; factor-analytic results show that specific survey items can be combined together into one index. Survey questions that loaded highly on the same factor were combined into a single scale by averaging the responses across items. Factor analyses of survey items exposed several indices, each measuring a general content area (e.g. loyalty, tech support).
  • 14. Copyright © 2010 Business Over Broadway Internal Consistency Reliability by Decision Influence Decision-Making Role INDEX Entire Sample Final Decision Influencer Purchasing 2009 Indices 1 Customer Loyalty Index* .89 .89 .89 .88 2 Perceived Value of External Information .85 .85 .83 .87 3 Perceived Importance of Doc/Mgmt/Reporting .80 .78 .79 .78 4 Perceived Importance of Ease of Use .79 .76 .80 .80 5 Quality of Security Products .91 .92 .91 .92 6 Quality of Tech Support .91 .91 .90 .91 2010 Indices 1 Customer Loyalty Index .82 .81 .83 .81 2 Perceived Importance of Doc/Mgmt/Reporting .70 .69 .66 .78 3 Perceived Importance of Ease of Use .66 .68 .64 .58 4 Perceived Quality of Security Products .89 .89 .88 .91 5 Perceived Quality of Tech Support .92 .93 .92 .95 Internal Consistency Reliability1 Major Points • Factor analyses of survey items exposed several indices, each measuring a general content area (e.g. loyalty, tech support). • Definition of index is based on the aggregated items • Internal consistency reliabilities are acceptable and similar across all decision- making categories. • Each index would be useful for executive dashboards since each provides reliable, useful information. 1 No significance testing possible for Cronbach’s alpha. *See previous slide for description of each index and the survey items that represent each index.
  • 15. Copyright © 2010 Business Over Broadway • Inter-contact reliability determines extent to which Contacts within an Account give similar customer satisfaction survey ratings – Indexed by a correlation between different Contacts within each Account across all Accounts • Inter-contact reliability addresses the consistency of the implementation of the customer satisfaction survey system across Contacts – Expect some degree of inter-contact reliability because Contacts are rating the same Account • Do not want perfect reliability – Perfect reliability would indicate no utility of surveying different management contacts within an organization • Calculate inter-contact reliability for each survey type across all business attributes Inter-Rater Reliability
  • 16. Copyright © 2010 Business Over Broadway Data Model for Inter-Rater Reliability Analysis Type of Contact Account 1 Account 2 Account 3 Account m Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Reliability Analysis Inter-Contact Reliability • Correlation of satisfaction ratings between different Contacts within Accounts • Calculated for each type of customer survey • Reliability could vary from 0 (no agreement between Contacts) to 1.0 (complete agreement between Contacts) . . . ... = Rater 2 Attribute Rater 1
  • 17. Copyright © 2010 Business Over Broadway Inter-Contact Reliability (Decision- makers vs. Influencers) INDICES1 Decision maker/ Influencer 1 Customer Loyalty Index* .19 2 Security Effectiveness -.12 3 Reporting .05 4 Management / Administration .15 5 Documentation -.04 6 Ease of Installation .16 7 End User Experience .02 8 Ease of Upgrade .22 9 Technical Support .05 10 Price .22 Major Points • There is low agreement between Contacts from different decision-making standpoint. • Contacts from different decision-making standpoints (e.g., Decision maker vs. Influencer) show low agreement in their ratings of COMPANY • The degree of disagreement across different survey respondents within an Account could reflect: •Poor communication within an Account •Different expectations of different functions •Different usage of COMPANY products 1 Survey data are from 2010 Survey; N = 47 to 50. No statistically significant correlations. *Customer Loyalty Index is composite of following four questions: Overall Satisfaction, Quality, Value and Recommend. Functional job function includes Finance, Not enough sample to examine “Purchasing” role.
  • 18. Copyright © 2010 Business Over Broadway • Test-retest reliability determines extent to which Contacts give similar ratings over time – Indexed by a correlation between ratings across two time periods – Higher correlation indicates that Contacts provide similar ratings over time • Addresses the consistency of ratings over time – If reliability is high, relative ordering of ratings do not tend to vary over time across different Accounts – Expect good degree of test-retest reliability because Contacts are rating the same Account over time • Contacts that are satisfied/dissatisfied at time 1 should also be satisfied/dissatisfied at time 2 • Contacts with high/low value ratings at time 1 should also have high/low value ratings at time 2 • Calculate test-retest reliability for satisfaction and value ratings Test-Retest Reliability
  • 19. Copyright © 2010 Business Over Broadway Data Model for Test-Retest Reliability Analysis Time 1 Survey Time 2 Survey Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Account 1 Account 2 Account 3 Account m Contact 1 Contact 2 Contact 3 Contact 4 Contact 5 Contact 6 Contact n-1 Contact n Reliability Analysis Test-Retest Reliability • Correlation of satisfaction ratings between same Contacts across two time periods • Correlation of satisfaction ratings (averaged over Contacts) between same Accounts across two time periods • Calculated only for Surveys that are repeated • Reliability varies from 0 (no agreement between time 1 and time 2) to 1.0 (complete agreement between time 1 and time 2) . . . . . . .. . = Contact n Contact n Time 1 Time 2
  • 20. Copyright © 2010 Business Over Broadway Test-Retest Reliability INDICES/MEASURES 1 Customer Loyalty Index .69 2 Overall Satisfaction .32 3 Quality .58 4 Value .52 5 Recommend .70 6 Security Effectiveness .43 7 Reporting .53 8 Management / Administration .54 9 Documentation .44 10 Ease of Installation .50 11 End User Experience .49 12 Ease of Upgrade .45 13 Technical Support .51 14 Price .54 Test-Retest Reliability1 1 Survey data are from 2009 (Time 1) and 2010 (Time 2). N = 472 to 478. All correlations significant at p < .01 level. CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct. = Contact n Contact n Time 1 Time 2 Major Points • Test-retest reliability is high • Customers tend to respond in a similar way over two time periods. • Those who are satisfied (dissatisfied) at time 1 tend to be satisfied (dissatisfied) at time 2.
  • 21. Copyright © 2010 Business Over Broadway Evidence of Validity
  • 22. Copyright © 2010 Business Over Broadway • Content validity is established when items in the survey are a representative sample of some defined universe of items • Universe of survey items represents all items that define the important elements of a company’s performance • Does the customer survey comprehensively assess customer requirements? • Must show “link” between set of items (survey) and the universe of items – Link determined by Subject Matter Experts (SMEs) Content-related Approach
  • 23. Copyright © 2010 Business Over Broadway • Disloyal customers provide comments regarding improvements1 • Comments/issues range from product to service issues • All issues/reasons covered by content of customer survey • Issues aligned with results of driver analysis Content-related Approach 1 Customer comments from those who rated Recommend question low.
  • 24. Copyright © 2010 Business Over Broadway • Criterion-related validity relies on statistical analysis rather than judgment • Established when scores on the survey are related to some other variable (criterion) already known to measure similar attributes • Key is developing/selecting right criteria to use in validation study – Measures of Account Quality – by Account Team – Objective Assessment Scores Criterion-related Approach
  • 25. Copyright © 2010 Business Over Broadway • Construct-related validity addresses what is being measured by examining relationships between constructs (e.g., customer satisfaction, customer loyalty, other measures of Account quality) • Concerned with theory and the proposed relationships between constructs based on that theory • Understanding the theoretical relationship of constructs to other constructs – To what should customer satisfaction be (convergent validity) and not be related (discriminant validity)? • Need to examine patterns of relationships using the survey and other instruments Construct-related Approach
  • 26. Copyright © 2010 Business Over Broadway 2009 Mean SD N 1 2 3 4 5 6 1. Customer Loyalty Index 4.84 0.94 2941 0.89 2. Quality of Security Product 4.59 0.86 2917 0.77 0.91 3. Quality of Technical Support 4.81 1.09 2880 0.68 0.69 0.91 4. Perceived Value of External Information 4.34 0.97 2787 0.31 0.37 0.29 0.85 5. Perceived Importance of Doc/Mgmt/Reporting 4.77 0.87 2914 0.29 0.43 0.31 0.34 0.80 6. Perceived Importance of Ease of Use 5.05 0.87 2919 0.33 0.48 0.33 0.28 0.54 0.79 Correlations Among Indices  Indices are correlated logically with each other  Higher ratings on key indices (quality, value) correspond to higher customer loyalty ratings compared to other indices (perceived importance) 2010 Mean SD N 1 2 3 4 5 1. Customer Loyalty Index 5.54 1.27 3449 0.82 2. Quality of Security Product 4.72 0.75 3417 0.78 0.89 3. Quality of Technical Support 4.95 1.04 3387 0.67 0.65 0.92 4. Perceived Importance of Doc/Mgmt/Reporting 5.01 0.66 3434 0.11 0.21 0.12 0.70 5. Perceived Importance of Ease of Use 5.17 0.69 3437 0.10 0.23 0.12 0.41 0.66 1 Cronbach’s alpha (internal consistency estimates) are located in the diagonal.
  • 27. Copyright © 2010 Business Over Broadway • Driver analysis shows that customers who are more satisfied also report higher levels of customer loyalty Construct-related Approach (2009) Mean Rating N Impact on Loyalty Customer Loyalty Index 4.84 2941 Understands Needs 4.57 2926 0.73 Addresses Needs 4.52 2924 0.78 Understands Better Than Competitors 4.30 2884 0.69 Security Effectiveness 5.18 2911 0.70 Reporting 4.40 2875 0.54 Management / Administration 4.52 2866 0.63 Documentation 4.23 2871 0.55 Ease of Installation 4.61 2884 0.58 End User Experience 4.66 2880 0.66 Ease of Upgrade 4.77 2856 0.61 Technical Support 4.80 2860 0.67 Price 4.35 2877 0.55 Compare COMPANY to Alternatives 4.06 2645 0.69 Tech Support via Telephone 4.96 1815 0.61 Tech Support via Email 4.79 1792 0.62 Tech Support via Web Knowledgebase 4.39 2102 0.49 1 No significance testing possible for Cronbach’s alpha. *CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
  • 28. Copyright © 2010 Business Over Broadway • Driver analysis shows that customers who are more satisfied also report higher levels of customer loyalty Construct-related Approach (2010) Mean Rating N Impact on Loyalty Customer Loyalty Index 5.54 3449 Understands Needs 4.32 3414 0.44 Addresses Needs 4.31 3408 0.48 Understands Better Than Competitors 4.37 3337 0.52 Security Effectiveness 5.21 3407 0.65 Reporting 4.58 3371 0.54 Management / Administration 4.67 3387 0.67 Documentation 4.48 3372 0.54 Ease of Installation 4.72 3382 0.56 End User Experience 4.73 3371 0.66 Ease of Upgrade 4.78 3356 0.59 Technical Support 4.91 3360 0.66 Price 4.62 3348 0.47 Compare COMPANY to Alternatives 4.01 3020 0.77 Tech Support via Telephone 5.05 2408 0.63 Tech Support via Email 4.91 2337 0.64 Tech Support via Web Knowledgebase 4.47 2585 0.55 1 No significance testing possible for Cronbach’s alpha. *CLI is a composite score consisting of four survey questions (Overall Satisfaction, Quality, Value, Recommend). Items were selected based on a factor analysis of survey questions; these four questions loaded highly on the same factor, suggesting that they measure the same construct.
  • 29. Copyright © 2010 Business Over Broadway • Use previous validity approaches (content and criterion) to support a theoretical model of customer satisfaction • Various theoretical models state that customer loyalty is impacted by satisfaction with product and satisfaction with service • Need to examine the pattern of relationships among these constructs • Pattern of correlations should support this model Construct-related Approach Service Satisfaction Product Satisfaction Customer Loyalty Basic Theory of Customer Satisfaction
  • 30. Copyright © 2010 Business Over Broadway • Test the model by examining correlations among three variables: – Product Quality (Quality of Security Eff.) – Tech Support Quality – Customer Loyalty (CLI) • Path analysis1 applied to observed correlations – Isolates degree of impact a given variable has on other variables • Analysis supports theory that both product quality and tech tech support quality impact customer loyalty independently Construct-related Approach CORRELATIONS CLI Tech Qual Product Quality .70 .53 Customer Loyalty (CLI) - .68 1 For detailed description of path analysis methodology, please see: Pedhazur, E. J. (1982). Multiple regression in behavioral research: Explanation and prediction (2nd Ed.) Holt, Rinehart and Winston. New York, NY. Tech Support Product Effectiveness Customer Loyalty Path Analysis .11 .54 .51
  • 31. Copyright © 2010 Business Over Broadway • Support Level and Customer Loyalty • Three levels of Support (Standard, Premium, Platinum) • Lower customer loyalty reported by customers with Platinum support – Platinum support customers could have higher expectations than Standard support customers. Support Level
  • 32. Copyright © 2010 Business Over Broadway • Difference found across regions1 – APAC has highest levels of customer loyalty and quality ratings – Southern Europe as lowest levels of customer loyalty and quality ratings Segmentation By Region 1 Based on survey results from 2010.
  • 33. Copyright © 2010 Business Over Broadway Segmentation By Industry Difference found across industries1 HED has the lowest levels of customer loyalty 1 Based on survey results from 2010.
  • 34. Copyright © 2010 Business Over Broadway Segmentation By License Tier Difference found across license tiers1 License tiers 3 and 4 have highest levels of customer loyalty and quality ratings 1 Based on survey results from 2010.
  • 35. Copyright © 2010 Business Over Broadway • Internal consistency reliability – Contacts tend to give similar ratings across the different items of the same scale • Non-perfect reliability indicates that different items still provide different information about customers’ attitudes • Reasons for extremely high reliability – High degree of redundancy of items – Rater fatigue due to survey length so respondents rate all questions similarly • Inter-contact reliability – Contacts within Accounts show little agreement in ratings – Different types of Contacts provide different vantage point when rating the company • Poor communication within Account across levels and/job functions • Different Contacts have differing expectations • Contacts are treated differently by Account team depending on their level/function Summary of Reliability of Survey Ratings
  • 36. Copyright © 2010 Business Over Broadway • Content-related approach – Questions in customer survey are representative sample of universe of questions • Items comprehensively cover customer requirements throughout all phases of the customer lifecycle • Dissatisfied customers’ open-ended comments are addressed by questions in the survey • Criterion-related approach – No external criteria are available at this time • Construct-related approach – Pattern of relationships of customer satisfaction data supports theory of customer satisfaction • Customer satisfaction is related to customer loyalty • Product quality impacts loyalty through perceived value Summary of Validity
  • 37. Copyright © 2010 Business Over Broadway • Self-assessment survey identified strengths and areas needing improvement1 • Overall, low to moderate adoption rates of best practices across all components of customer feedback programs – Customer feedback not as important as financial metrics, customer feedback not integrated with CRM system, problem resolution not integrated into CRM system • Strengths seen in specific areas of COMPANY’S customer feedback program: – Top executive is champion of the program, multiple methods of data collection, Web surveys, various measures of customer loyalty, customer research presented externally, existing customer data used for segmentation Customer Feedback Programs Best Practices Survey 1 Survey identifies extent to which company adopts best practices in their customer feedback program. EMPLOYEE JANE DOE completed the online survey (http://www.businessoverbroadway.com/cfpbpi.htm) for COMPANY on June 10, 2010. See survey for questions.
  • 38. Copyright © 2010 Business Over Broadway • The company’s survey process produces reliable, valid indices of customers’ attitudes • Evidence of reliability – All indices are highly reliable – items within indices can be combined to create summary scores • Evidence of validity – Survey questions comprehensively assess important areas to the customers – Each index measures unique construct apart from the other indices – Relationships among indices supports customer satisfaction models Conclusions
  • 39. Copyright © 2010 Business Over Broadway • The company can feel confident using customer survey results when making business decisions – Identifying drivers of customer loyalty – Using Indexes to manage customer relationships • Understand why different customers within the same Account hold different attitudes toward COMPANY – Do they communicate with each other (through Company-sanctioned User groups or individually)? They could get together and talk about the benefits of COMPANY. Could COMPANY assist in this process? Conclusions
  • 40. Copyright © 2010 Business Over Broadway • Changes in Customer Loyalty Measurement – Put loyalty question at start of survey (to assess top of mind responses) with common rating scale (recommend 0 to 10) scale) – Add questions to measure other components of customer loyalty • Purchasing Loyalty: Example: How likely are you to purchase additional/different types of products from COMPANY? • Retention Loyalty: Example: How likely are you to switch to a different security provider within the next 12 months? • Benefits – Make decisions to grow business through new and existing customers – Loyalty indices can be used to build customer-centricity • Executive dashboards, guide strategy/governance, resource allocation, incentive compensation, benchmark against industry standards Recommendations on Method
  • 41. Copyright © 2010 Business Over Broadway • Changes in Technical Support Measurement – Move all technical support questions into one section • Telephone, Email, and technical support – Consider adding other technical support questions to broaden technical support coverage • Knowledge, professionalism, care • Benefits – Easier for customers to complete survey – Provide more specific feedback to technical support personnel • Consider ways to increase response rates – Email invitation content Recommendations on Method
  • 42. Copyright © 2010 Business Over Broadway • Integrate customer feedback into company’s strategy/governance1 – Company’s strategic vision, mission and goals; executives’ objectives and incentive compensation, front-line employees’ objectives and incentive compensation • Integrate customer feedback into daily business processes1 – CRM system; company/executive dashboards; communicate results regularly to the entire company • Conduct in-depth research using customer survey results1 – Linkage analysis between customer feedback and operational metrics, financial metrics, employee metrics. Recommendations on Strategy, Integration and Research 1 Based on Customer Feedback Programs Self-Assessment Survey Results.
  • 43. Copyright © 2010 Business Over Broadway • Consider other “financial” metrics to track and link to customer survey – Consult with CFO to consider other financial metrics that are tracked and consider using those to identify which aspects of customer satisfaction/loyalty relate to financial growth of COMPANY • e.g. Renewals – Track financial metrics over time in conjunction with the customer survey to conduct longitudinal studies • Study impact of attitudes at time 1 on financial metrics at time 2 and beyond • Linkage study between COMPANY’S transactional survey and relationship survey – To what extent do transactions (e.g. in call center) impact long- term customer loyalty Recommendations for Future Research
  • 44. Copyright © 2010 Business Over Broadway For More Information Business growth through customer insight Business Over Broadway Bob E. Hayes, Ph.D. Email: bob@businessoverbroadway.com Web: www.businessoverbroadway.com Blog: www.businessoverbroadway.blogspot.com Twitter: www.twitter.com/bobehayes