Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

What Your Customers Really Think About You (Relate Live Sydney)

Lori Gauthier, Ph.D., Director of Marketing Research, Zendesk

In this workshop, you’ll learn how to design survey questions that reveal what people are really thinking. Attendees will leave the workshop with two surveys that quickly and accurately measure customer satisfaction (CSAT) and customer effort (CES) -- surveys that can be used by any organization whether they serve customers, employees, students, volunteers, vendors, or the general public. Lori will review the key do's and don'ts of designing methodologically sound surveys, helping you avoid data-destroying random error and bias and get to the answers you need. Better data. Better relationships.

Related Books

Free with a 30 day trial from Scribd

See all
  • Be the first to comment

  • Be the first to like this

What Your Customers Really Think About You (Relate Live Sydney)

  1. 1. #RelateLive
  2. 2. Lori Gauthier, Ph.D. Zendesk Director of Marketing Research @datadocgauthier
  3. 3. #RelateLive What Your Customers Really Think About You
  4. 4. Let’s Start with the Don’ts!
  5. 5. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree construct not specified in scale unbalanced question non-modified response options agree/disagree scale question as a statement missing ambivalent midpoint incorrectly defined construct
  6. 6. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort missing correct midpointmissing “no effort” end point confusing scale incorrectly defined construct awkward question Measuring Customer Effort What’s wrong with this question?
  7. 7. Measurement Error The survey itself impacts responses specification error random error systematic error largest source of error controlled by surveyor
  8. 8. “I know you think you understand what you thought I said but I'm not sure you realize that what you heard is not what I meant” - Unknown
  9. 9. Wait. What? I Thought You Meant… Specification Error Even well designed surveys can yield bad data when the wrong constructs are measured or the right constructs aren’t measured completely.
  10. 10. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree incorrectly defined construct (specification error)
  11. 11. What’s wrong with this question? Measuring Customer Effort Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort incorrectly defined construct (specification error)
  12. 12. Stewie Data Look at him go! Random Error Bad survey design can introduce data- destroying random error, making your data — and decisions — bounce all over the place.
  13. 13. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree construct not specified in scale (random error) non-modified response options (random error) agree/disagree scale (random and systematic error)
  14. 14. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort awkward question (random error) confusing scale (random error) missing correct midpoint (random error) Measuring Customer Effort What’s wrong with this question?
  15. 15. Rooting Out Random Error So long, Stewie! no! nooo! noo! double barreled question unexpected scale direction insensitive scale overly sensitive scale scale without midpoint scale without verbal labels overlapping scale labels non construct-specific scale confusing question or scale true|false, yes|no, agree|disagree scale
  16. 16. Tower of Pisa Data One way or another, it’s gonna getcha! Systematic Error Bad survey design can introduce data-destroying systematic error, leading you to make biased decisions.
  17. 17. What’s wrong with this question? Measuring Customer Satisfaction How much do you agree with the following statement? I am happy with the customer support I received today. Strongly disagree Strongly agree Disagree Agree Somewhat disagree Somewhat agree unbalanced question (systematic error) agree/disagree scale (random and systematic error) question as a statement (systematic error) missing ambivalent midpoint (systematic error)
  18. 18. Question source: The Effortless Experience How much effort did you personally have to put forth to get your issue resolved? Very low effort Very high effortNeutral High effortLow effort missing “no effort” end point (systematic error) Measuring Customer Effort What’s wrong with this question?
  19. 19. Banishing Bias Arrivederci, Pisa! worst ever!!! thingunbalanced scale leading question true|false, yes|no, agree|disagree scale missing extreme endpoints bipolar scale without neither/nor midpoint order effects context effects unbalanced question question formatted as statement
  20. 20. Done with the Don’ts. Let’s Review the Do’s!
  21. 21. Know What You Need from Your Data answer construct question scale Start with your destination.
  22. 22. What Are You Measuring? Are You Sure?
  23. 23. Define What You Need to Measure Words Mean Things Search definitions, synonyms, antonyms. Use the language and tone appropriate for your population. Result: Respondents answer the question you think you’re asking.
  24. 24. Source: snappywords.com
  25. 25. What Questions and Scales Should You Use? Understanding Construct Polarity and Scale Sensitivity
  26. 26. Which Way Do We Go? Construct polarity Unipolar Construct Bipolar Construct Very common; typically specific; often descriptive Very rare; typically global; occasionally comparative Measures absence to maximum: not at all likely to extremely likely Measures maximum negative to maximum positive: disapprove a great deal to approve a great deal Midpoint represents half of construct Midpoint represents ambiguity or no opinion 5-point scale is ideal 7- or 9-point scale is ideal How likely are you to vote in a primary this year? Do you approve or disapprove of negative campaigning? Examples: likelihood, frequency, duration, intensity Examples: bad/good, dis/satisfied, dis/like, worse/better common labels: not at all, slightly, moderately, very, extremely none, a little, a moderate amount, a lot, a great deal common labels (mirrored sides): extremely, very, moderately, slightly, neither/nor … a great deal, a lot, a moderate amount, a little, neither/nor … zero ????
  27. 27. Ideal scale sensitivity (example 1) How Many Scale Points Should You Use? unipolar notatall extrem ely m oderately slightly very 1000 5025 75 bipolar neither/nor extrem ely m oderately slightly very 1000 5025 75 slightly very extrem ely m oderately -25-75-100 -50
  28. 28. Ideal scale sensitivity (example 2) How Many Scale Points Should You Use? unipolar notatall a greatdeal a m oderate am ount a little a lot 1000 5025 75 bipolar neither/nor a greatdeal a m oderate am ount a little a lot 1000 5025 75 a little a lot a greatdeal a m oderate am ount -25-75-100 -50
  29. 29. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed unipolar not at all likely extremely likely moderately likely slightly likely very likely 1000 5025 75 ???? not likely likely
  30. 30. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount
  31. 31. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed neither like nor dislike like a great deal like a moderate amount like a little dislike a little dislike a great deal dislike a moderate amount 1000 33 67-33-67-100 bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount
  32. 32. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount 1000 50-100 -50 neither like nor dislike like a great deal like a moderate amount dislike a great deal dislike a moderate amount
  33. 33. How Many Scale Points Should You Use? Sensitivity reduced as scale points removed bipolar 1000 5025 75-25-75-100 -50 neither like nor dislike like a great deal like a moderate amount like a little like a lot dislike a little dislike a lot dislike a great deal dislike a moderate amount 1000-100 neither like nor dislike like a great deal dislike a great deal
  34. 34. Measuring Customer Satisfaction
  35. 35. How satisfied are you withAcme’s customer support? 1 3 42 What’s wrong with this question? Measuring Customer Satisfaction
  36. 36. What’s wrong with this question? Measuring Customer Satisfaction How satisfied are you withAcme’s customer support? 1 3 42 unbalanced question (systematic error) missing construct- specific verbal labels (random error)missing negative half of scale (systematic error) missing midpoint on positive half of scale (random error) missing zero scale point (systematic error)
  37. 37. A methodologically sound question Measuring Customer Satisfaction Overall, how satisfied or dissatisfied are you withAcme’s customer support? moderately dissatisfied slightly dissatisfied neither satisfied nor dissatisfied slightly satisfied moderately satisfied extremely dissatisfied extremely satisfied 7-point, fully labeled, construct-specific, bipolar scale measures what we want to measure: satisfaction with customer support “overall” appropriate for global-level measure balanced question ambivalent midpoint
  38. 38. Measuring Customer Effort
  39. 39. To what extent do you agree or disagree with the following statement? The company made it easy for me to handle my issue. Strongly disagree Strongly agree Neither agree nor disagree Disagree Agree Somewhat disagree Somewhat agree Question source: The Effortless Experience What’s wrong with this question? Measuring Customer Effort
  40. 40. To what extent do you agree or disagree with the following statement? The company made it easy for me to handle my issue. Strongly disagree Strongly agree Neither agree nor disagree Disagree Agree Somewhat disagree Somewhat agree Question source: The Effortless Experience construct not specified in scale (random error) non-modified response options (random error) agree/disagree scale (random and systematic error) question as a statement (systematic error) What’s wrong with this question? Measuring Customer Effort
  41. 41. Measuring Customer Effort A methodologically sound question How easy was it to get the help you needed from us today? not at all easy extremely easy moderately easy very easy slightly easy measures what we want to measure: effort needed to get company’s help “today” appropriate for transaction-level measure 5-point, fully labeled, construct-specific, unipolar scale
  42. 42. Measuring Customer Effort What is driving customer effort? Content source for drivers of effort: The Effortless Experience How did we make it difficult? (Check all that apply) You didn’t solve the problem I had to contact the company multiple times I felt like I was talking to a robot I had to repeat myself I had to use a channel I don’t like (phone, web form, chat, email, FAQ) I was transferred from person to person Some other reason (Please specify) don’t assume resolution pick list Q measures frequency of known drivers open-ended option captures unknown drivers limit list to 7-9 options random rotate pick list
  43. 43. Sound design. Accurate data. Better relationships. A Step-by-Step Approach to Survey Design start at your destination define your construct draft question + scale check for random error check for systematic error collect accurate data bing! bing! bing! determine polarity
  44. 44. Thank You! Questions? Contact me at lgauthier@zendesk.com or @datadocgauthier.
  45. 45. #RelateLive

    Be the first to comment

    Login to see the comments

Lori Gauthier, Ph.D., Director of Marketing Research, Zendesk In this workshop, you’ll learn how to design survey questions that reveal what people are really thinking. Attendees will leave the workshop with two surveys that quickly and accurately measure customer satisfaction (CSAT) and customer effort (CES) -- surveys that can be used by any organization whether they serve customers, employees, students, volunteers, vendors, or the general public. Lori will review the key do's and don'ts of designing methodologically sound surveys, helping you avoid data-destroying random error and bias and get to the answers you need. Better data. Better relationships.

Views

Total views

1,366

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

15

Shares

0

Comments

0

Likes

0

×