The document provides essential tips for enhancing survey quality, emphasizing the use of 5 or 7-point Likert scales and the importance of breaking down broad concepts into multiple specific questions. It also warns against biases like priming and order effects, suggesting randomization of questions to mitigate these issues. Additionally, the importance of tracking changes in survey responses over time to measure member satisfaction and happiness is highlighted.

1. Tips to radically improve your
Surveys
@stephbeadell

2. Stephanie Beadell
VP of Marketing
Stephanie@Briggsby.com
@StephBeadell

4. ? + =
@stephbeadell

5. Surveys are Powerful Tools

6. …unless the data is bad.
@stephbeadell

7. Radically Improve Your Data

8. 3 @stephbeadell
steps

9. Typical Survey Question
@stephbeadell
Do you feel like part of our community?

10. Do you feel like part of our community?
@stephbeadell
Yes
No
I Don't Know

11. 1 @stephbeadell

12. Use a 5 or 7-point
Scale
@stephbeadell

13. A Better Question
@stephbeadell
I fit in with the community.
Strongly
Agree
Strongly
Disagree
Neutral

14. I Fit in with the Community
@stephbeadell
0
20
40
60
80
100
120
140
160
180
200
Strongly
Disagree
Disagree Neutral Agree Strongly
Agree
I Fit in with the Community
Average: 3.67

15. “I Fit in with the Community” Score Over Time
@stephbeadell
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
I Fit in with the Community

16. “Likert Scale”
@stephbeadell

17. Likert Scale Basics
@stephbeadell
Strongly
Agree
Neutral
Strongly
Disagree
5 or 7 points

18. Likert Scale Basics
@stephbeadell
Brave
Shy
Unipolar
(Don’t Do This)

19. Analysis
Opportunities
@stephbeadell

20. Likert Scale Question
@stephbeadell
I fit in with the community.
Strongly
Agree
Strongly
Disagree
Neutral

21. Scoring the Answers
@stephbeadell
I fit in with the community.
1 2 3 4 5

22. Correlation Analysis & More
@stephbeadell
“Heavy forum users are more likely to
be satisfied with their membership.”
Satisfaction
Frequency of Forum Visits
Forum Usage vs. Satisfaction

23. Use a 5 or 7-point
Scale
When in Doubt:
@stephbeadell

24. 2 @stephbeadell

25. @stephbeadell
into multiple questions
Break down
Big Concepts

26. Typical Survey Question
@stephbeadell
Are you happy with the community?

27. … it could mean
@stephbeadell
Do you like the people in the community?

28. … it could mean
@stephbeadell
Does the community meet your needs?

29. … it could mean
@stephbeadell
Does being a member make you feel good?

30. @stephbeadell
?
?
? ?

31. Try this instead
@stephbeadell
1. Being a member is valuable to my career.
2. I enjoy being a member.
3. The company cares about the community.

32. Report it Like This
@stephbeadell
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
5
Valuable to
Career
Enjoy Being
Part
Company
Cares
Member Happiness Survey
Average Happiness
Score:
11 out of 15

33. Track it Over Time
@stephbeadell
“Our team increased member happiness by 3 points this quarter.”
0
2
4
6
8
10
12
14
16
Month 1 Month 2 Month 3
Happiness

34. @stephbeadell
Break down
Big Concepts

35. 3 @stephbeadell

36. Typical Survey Results
@stephbeadell
0
20
40
60
80
100
120
140
160
180
200
Strongly
Disagree
Disagree Neutral Agree Strongly
Agree
I Fit in with the Community2.
Freelancer
Small
Business
Enterprise
Business Size1.

37. Compare and
Contrast
@stephbeadell

38. Comparing Answers
@stephbeadell
EnterpriseSmall Business
0
10
20
30
40
50
60
70
80
I Fit in with the Community
0
10
20
30
40
50
60
70
80
I Fit in with the Community

39. Typical Survey Results
@stephbeadell
0
20
40
60
80
100
120
140
160
180
200
Strongly
Disagree
Disagree Neutral Agree Strongly
Agree
I Fit in with the Community
2.
Freelancer
Small
Business
Enterprise
Business Size1.

40. Comparing Answers
@stephbeadell
EnterpriseSmall Business
0
10
20
30
40
50
60
70
80
I Fit in with the Community
0
10
20
30
40
50
60
70
80
I Fit in with the Community

41. How to Do It in Excel: Pivot Tables
@stephbeadell
Export Raw Data

42. And in SurveyMonkey: Filter/Compare
@stephbeadell
Filter or Compare

43. @stephbeadell
Plan segments in advance

44. Comparing Answers
@stephbeadell
Ask identifying questions

45. 3 @stephbeadell
steps

46. Use a 5 or 7-point
Scale
When in Doubt:
@stephbeadell

47. @stephbeadell
Break down
Big Concepts

48. Compare and
Contrast
@stephbeadell

49. How Not to Screw it Up

50. @stephbeadell
Bias
Introducing

51. @stephbeadell
Priming

52. Bargh, Chen, and Burrows (1996)
@stephbeadell
Sentence Scramble Task
Rude
aggressively, bold, rude, bother, disturb, intrude, annoyingly, interrupt,
audaciously, brazen, impolitely, infringe, obnoxious, aggravating, bluntly
Polite
honor, considerate, appreciate, patiently, cordially, yield, polite, cautiously,
courteous, graciously, sensitively; discreetly, behaved, unobtrusively
Neutral
exercising, flawlessly; occasionally rapidly, gleefully, practiced,
optimistically, successfully, normally, send, watches, encourages. gives,
clears, prepares

53. Bargh, Chen, and Burrows (1996)
@stephbeadell
People who Interrupted After the Task
Polite Neutral Rude

54. Introducing Bias: Priming
@stephbeadell
When are people prompted
to take the survey?

55. @stephbeadell
Order Bias

56. Order Bias
@stephbeadell
How important is content marketing?

57. Order Bias
@stephbeadell
How important is content marketing?
How much content marketing do
you plan to do this year?

58. Solving Order Bias with Randomization
@stephbeadell
Question Randomization

59. Solving Order Bias with Randomization
@stephbeadell
Shuffle Question Order

60. @stephbeadell
Stereotype
Threat

61. Stone, Lynch, Sjomeling, and Darley (1999)
@stephbeadell
“natural athletic ability”
“sports intelligence”

62. Inzlicht (2000)
@stephbeadell
Female Math Test Performance When Others Present
2 Women 2 Men

63. Solving for Stereotype Threat
@stephbeadell
Demographic data always
goes at the end of a survey

64. Thanks!