Presentation by Ted Hopton at JiveWorld 2017

1. Data-Driven Decisions
with DES & Surveys
Ted Hopton| Director of Social Business
McGraw-Hill Education
May 2017

2. My Community
2
Culture, Communication, Learning
“Spark will become the connective tissue that reaches and engages
people throughout the organization.”

3. My Community Data Journey
3
In the beginning…
2003: Customer Community - Page Views
2008: Jive v2.5 internal community at
UBM - Crude & Rudimentary data

4. My Community Data Journey
4
2010: Business Objects
– Built my own tools

5. My Community Data Journey
5
2012: CMR & Excel
– Built my own

6. My Community Data Journey
6
2014: Internal community at MHE - Participation Ladders (in Excel)

7. My Community Data Journey
7
2015 -
present:
DES &
Tableau
– Building
& defining
my own
analytics
– See my
JiveWorks
profile,
Featured
Content

8. Improving
Search with Jive
DES & Surveys

9. Mission: Fix Search
9
Problem: Vague complaints about finding
things in the community
 Search should work better
Solution: Educate people?
Challenge: Don’t know what the actual
problem is…
Hypotheses:
 Search more often -> happier with search
 Spotlight searchers are happier

10. DES & Survey
10
Jive Data Export Service (DES) Report
 Users & number of searches
 >= 900 searches in past year
 1925 informed people to survey

11. Short Survey
11

12. Short Survey
12

13. Short Survey
13
Net Promotor Score (NPS)

14. Survey Results
14
Turns out, asking people to rank things is hard
 Some even thought the survey tool was broken
Only got 163 responses (8% response rate)
 Still a lot better than guessing blindly, though
How did my hypotheses hold up?
 Not so well… so much for my hunches…

15. NPS by Average Number of Searches
15

16. Hypotheses Disproved
16
No clear correlation between search
activity and satisfaction
No difference in NPS between
Spotlight Searchers and hitting enter
Disproved both hypotheses
 But now we knew what was NOT true
 Did not waste time on convincing people
to use spotlight search
 Still not sure how to solve problem

17. Online Form: Can’t Find It? We’ll Help
17
Offered help
 Education/awareness
 Address need
 Quick response
• Teach that person
 About 2/week
Collected more data
 Track inquiries
 Record responses &
action taken
 Analyzed the data…

18. Content Failures, Not Search Failures
18
Content not in the community
 Had wondered about this…
 Actually 39% of requests!
 Not a search failure, but a
content failure
 Opportunity to improve
Or it’s hard to find
 Make it findable
 Title, tags, location, content
type, promote
 Remove outdated versions

19. Sales Can’t Find Things in Spark
19
January 2017 Higher Education Sales
Leaders Meeting
 People are very unhappy
 “Can’t find things”
 “Lots of outdated stuff”
Improve Spark = Top Goal
 Fix search
 Make sales happy – they need to bring
in the revenue

20. Very Short Survey
20
Measurement Requires a Baseline

21. Ask and Ye Shall Receive
21
Only 1 required question
 Directly asked, instead of NPS
 Only a 5-point scale
 Will survey again to measure
progress
Solicited specific examples
 Messy, hard to quantify
 Better than guessing or assuming
Invitation to self-identify
 Tie back to DES data
 Follow-up to learn more
DES to invite participants
 Had actually used the site

22. Search Does Not Suck for Everyone
22
Really short surveys work
 1600 responses = 27% response rate
 1000 gave examples & 350 gave their names

23. Unexpected Insight: Business Unit Differences
23

24. Search Sucks for Sales
24
Higher Ed Sales is REALLY unhappy
 Average rating 37% lower than company’s overall
 21% lower than their non-sales colleagues in Higher Ed

25. Why Organizational Differences?
25
Previous survey revealed what people search for most
 "Information, answers or help I need to do my job”
 57% top reason
 87% one of top two reasons
Hypothesis: It’s a content problem
 Different parts of the business do a better job than others of providing and
organizing content that people need to do their jobs
 Hence different levels of satisfaction when searching in Spark
Open-ended responses confirmed
 Many specific examples of frustration finding job-related content
 Sales can’t find: case studies; reports; pricing, product and implementation info;
training materials, integration information

26. Back to DES to Dig Deeper
26

27. Investigate in the Community
27
Evaluated HE Sales’ top 20 Places
 Commendations
 Recommendations
Evaluated HE Sales’ top 100 content
 Categorized
 Analyzed opportunities to improve
Conclusion:
 Missing content
 Outdated content
 Poorly organized content

28. Data Pointed to the Action Needed
28
Improve the content
 Work with management, group owners & authors
 Fill gaps
 Improve quality
 Information architecture/organization
• Point to content outside of community, too
 Mobile-friendly
Get user feedback
 Review prototypes
 Re-survey
• Measure progress
• Collect more data
 Continual process…
Apply across the community

29. After Making Improvements
29
New page design
 Aligned with
goals
 Mobile-friendly
 Visual CTA’s go
to link-filled
documents
 Streamlined
 Invites
interaction
Remember:
 People said “I
can’t find stuff”

30. Data-Driven Decision Making Takeaways
30
1. Don’t trust your gut. Get data.
2. Getting data isn’t hard. Use surveys.
3. Data analytics tools support data-driven decision-making.
4. It’s a journey: solutions to hard problems rarely reveal themselves
right away. Keep asking questions and seeking answers.

31. Latest Results
31
Asked Higher Ed Sales for an Update ... Up to the minute results…
 68 responses
 Improved results, even though we don’t feel we’ve accomplished much yet

32. Follow Up & Learn More
32
Contact:
 Ted.Hopton@gmail.com
 In JiveWorks
 Connect & share ideas at JiveWorld
Advanced Customer Measurement in JiveWorks
 Rich set of resources
 Ask questions
 Collection of my posts
• Using Jive DES: An Evolving Exploration of Community
Measurement Analysis
• Featured Content on my JiveWorks profile

33. Questions?