This document summarizes a benchmarking collaboration between Higher Logic and Marketing General to identify best practices for member engagement. It analyzed engagement data from over 400 organizations to create a composite engagement score measuring variables like subscribers, messages, and response rates. The data is broken down by organization size and quartiles of engagement. Higher scores correlated with improved membership retention. The goal is to help organizations understand their engagement statistics compared to peers and identify areas for growth.

1. Make an Impact with
Benchmarking, Analytics
and Engagement Reporting
A Benchmarking Collaboration between Higher Logic and Marketing General
Andy Steggles – President & Chief Customer Officer

2. Goal is to Identify:
• Best Practices for Engaging Members
• Quantify Benchmarking Metrics for Different Sized
Organizations
• Engagement Potential
• Correlation Between Engagement and Satisfaction

3. Methodology
• Perform Correlation Analysis
• Create a Composite Engagement Score (CES)
• Compare with 2014 MGI MM Benchmark
• 54 Engagement Variables (plus Ratios Between
Variables)

4. 2013 Engagement Data Analyzed from:
• 20,314,584 Users (12m qualified)
• 6,607,897 Members (4m qualified)
• 2,376,321 Subscribers (1.8m qualified)
• 2,205,496 Members (1.6m qualified)
• 170,825 Non-Members (154k qualified)
• 419 Orgs (256 qualified)
• Opted In (not anonymous)
• 97 Qualified Orgs and 19 Non-Qualified Orgs
4

5. Timeframe and Assumptions
• Point in Time Data for Some Elements
• Focus is on Subscribers vs. Members
• Date range is 2013 (where applicable)
• Size Categories by Number of Members:
Small 0 - 1,499
Small/Medium 1,500 - 4,999
Medium 5,000 - 19,999
Medium/Large 20,000 - 49,999
Large 50,000 – 99,999
X-Large 100,000+

6. Inclusion Criteria
• Org must have ALL of the below to qualify. Greater
than:
• 50 members
• 50 subscribers
• 50 messages
• 0.005 Message per Member Ratio
• Organizations that have posted their 100th
message after 1/1/2013 were flagged in the CES
ranking because they had less than a full year of
engagement maturity.

7. Every Organization has
different goals. What is
important to your
organization?
7
Photo credit: jepoirrier

8. 681,209 Discussion Messages
• 238,333 Threads (avg 931)
• 93,045 with Replies to Group (avg 363)
• 62,738 with Replies to Sender (avg 246)
• 114,759 with Replies to Group or Sender (avg 448)
• 442,242 Replies (avg 1,728)
• 328,259 to Group (avg 1,282)
• 114,617 Replies to Sender (avg 448)
• 412,127 to Group or Sender (avg 1,610) (where there is at least one
group reply)

9. 111,346 Authors
• 61,825 Thread Authors (avg 242)
• 30,943 Created Only One Thread (avg 121)
• 30,882 Created More than One Thread (avg 121)
• 62,928 Replies to Group Message Authors (avg 246)
• 30,978 Replied Only Once (avg 121)
• 31,950 Replies More than Once (avg 125)
• 44,847 Replies to Sender Authors (avg 175)
• 26,993 Replied Only Once (avg 105)
• 17,854 Replies More than Once (avg 70)

10. Success is Subjective
• Most Common Discussion Success Metrics
• Number of Discussion Messages (Activity)
• Percentage of Members Subscribed (Reach)
• Number of Responses per Thread (Value)
• Number of Authors (Distribution)
• What is Important to Your Organization?

11. "In most online communities, 90% of
users are lurkers who never contribute,
9% of users contribute a little, and 1% of
users account for almost all the action."
- Jakob Nielsen (2006)
Nielsen Norman Group
11
What are our stats?
• 1.8m subscribers
• 111k authors

12. What are our stats?
• 1.8m subscribers
• 111k authors
Result: 94-6

13. Average Number of Users, Subscribers and Contributors by Size
*In some cases, subscribers may include non-members.
**Example of “Ratio of Subscribers per Member”: The ratio for a small organization is 0.84. This means that there are 84
subscribers for every 100 members, while a Large Associations with a ratio of 0.21 means there are 21 subscribers for every
100 members.

14. Of those who Contributed, percentage who:

15. Bios and Photos
• Subscribers with Bios are 28% more likely to author
threads than an average subscriber.
• Subscribers with Photos are 5% more likely to
author threads than an average subscriber.

16. Threads
*For threads with at least one group reply or reply to sender
**For threads with at least one group reply
***For threads with at least one reply to sender

17. Percentage of Threads with at least one:

18. Thread Statistics Broken out by Organization Size*
Percentage of Threads with at least one:
• *Size Determined by Total Number of Members
• Note: Groupings do not add up to 100 because the
threads without a reply is not displayed.
Small 0 - 1,499
Small/Medium 1,500 - 4,999
Medium 5,000 - 19,999
Medium/Large 20,000 - 49,999
Large 50,000 – 99,999
X-Large 100,000+

19. Composite Engagement Score (CES)
• CES Formula: CES =
[(ratio of subscribers to members x weight) +
(ratio of messages to members x weight) +
(ratio of responses per thread x weight)] x 100
• Ratio of Subscribers to Members tells us on average how many subscribers per each member there is in an
organization
• Ratio of Messages per Member tells us on average how many messages a member generates
• Ratio of Responses per Thread tells us on average how many responses each thread

20. Who has the Better Performance? (Large Orgs)
OrgName TNOM TNOS TNOT TNORTG TGM CES
American Counseling Association
Note: ACA went live in June 2013
58,552 46,503 835 1,736 2,571 17.91687
Institute of Management Accountants
66,365 30,372 1,565 1,965 3,530 13.41126
National Association of College and University Business
Officers
53,826 13,168 997 1,474 2,471 12.73087

21. Who has the Better Performance? (Medium Orgs)
OrgName
TNOM TNOS TNOT TNORTG TGM CES D100MP
American Academy of Periodontology (AAP) 7,701 6,890 349 747 1,096 27.28743 2013-06-17
Construction Financial Management Association 6,793 6,658 309 625 934 27.27245 2013-01-21
National Business Officers Association 9,967 2,285 399 1,350 1,749 26.86057 2009-09-09
Dynamics NAV User Group 7,106 5,722 345 642 987 25.83747 2011-03-18
Dynamics AX User Group 15,556 12,552 747 1,355 2,102 25.27181 2013-03-26
The Institute of Continuing Legal Education 11,768 8,863 733 1,095 1,828 25.20535 2012-09-21
Regulatory Affairs Professionals Society 11,947 10,678 458 911 1,369 24.72974 2011-06-21

22. Who has the Better Performance? (Small Orgs)
OrgName TNOM TNOS TNOT TNORTG TGM CES
Alliance of Comprehensive Planners 242 236 1,808 4,056 5,864 2195.70
Avectra Users Group 1,095 547 788 1,136 1,924 167.83
The ALS Association 392 381 205 269 474 120.25
National Association of State Procurement Officials 1,190 1,260 612 491 1,103 92.43
League of Historic American Theatres 921 754 276 480 756 86.67

23. Discussion Activity by Quartile

24. Of those who Replied to Group, percentage who:

25. Average Composite Engagement Score (CES) by
Organization Size

26. Average Composite Engagement Score (CES)
by Quartile

27. Average Composite Engagement Score (CES)
broken out by Organization Size and Quartile

28. Discussion Based Email Open Rates
• Average Daily Digest Open Rate: 26.9%
• Average Real Time Open Rate: 34.13%

29. Correlation of Renewal & Engagement
Associations by 2014 MGI MMBR & Associations with an Engaged Online
Community who Agreed to Participate in a Blind-Comparison
IMO TRADE
2014 MGI Membership &
Marketing Benchmark Report
76% 85%
Associations with Engaged Online
Community
79% (+3%) 92% (+7%)
% of Improved Retention Correlated
with Online Engagement 5%

30. Example: American Society of
Association Executives (ASAE)
• Use Net Promoter Methodology
• 21,533 members
• 13,272 subscribed to Collaborate
• 3016 Completed Survey
• 81.6% (2462) of the Respondents were Collaborate
Users

31. The Hall of Fame
OrgName
TNOM TNOS TNOT TNORTG TGM CES Size Category
Alliance of Comprehensive Planners 242 236 1,808 4,056 5,864 2195.70 Small
National Employment Lawyer's Association 2,146 1,998 4,874 6,230 11,104 476.74 Small/Medium
American Society of Ophthalmic Administrators 2,640 2,562 3,181 7,731 10,912 386.85 Small/Medium
Piano Technicians Guild 3,579 3,377 2,970 11,543 14,513 379.67 Small/Medium
Veterinary Hospital Managers Association 2,170 2,036 1,240 3,796 5,036 223.57 Small/Medium
International Association of Rehabilitation Professionals 2,661 2,570 2,000 3,949 5,949 215.90 Small/Medium
National Society of Accountants 9,438 9,525 3,027 14,997 18,024 186.88 Medium
Avectra Users Group 1,095 547 788 1,136 1,924 167.83 Small
Higher Logic Users Group 2,837 1,235 1,272 2,849 4,121 142.86 Small/Medium
University Risk Management and Insurance Association, Inc., 2,178 2,546 1,156 1,920 3,076 140.41 Small/Medium
The ALS Association 392 381 205 269 474 120.25 Small
The American Association of Nurse Assessment Coordinators 13,553 13,567 6,735 9,065 15,800 116.65 Medium

32. Your Personalized Benchmarking Data

33. Andy Steggles
President & Chief Customer Officer
andy@higherlogic.com