Community Metrics at Novell
Upcoming SlideShare
Loading in...5
×
 

Community Metrics at Novell

on

  • 1,423 views

 

Statistics

Views

Total Views
1,423
Views on SlideShare
1,418
Embed Views
5

Actions

Likes
2
Downloads
11
Comments
0

2 Embeds 5

http://www.slideshare.net 4
http://ricetopher.tumblr.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Community Metrics at Novell Community Metrics at Novell Presentation Transcript

  • Community Metrics The Novell Approach Lee Romero http://blog.leeromero.org 2009 October
  • Contents
    • Communities at Novell 3
    • Setting up the Discussion 6
    • Membership Metrics 10
    • Activity Metrics 17
    • Tying into Performance Mgmt 22
    • Some Advanced Metrics 25
    • Additional Metrics 28
    • Final Words 32
  • Communities at Novell
  • Community Metrics at Novell A brief overview of the program, part 1
    • Novell started a CoP program in 2002/2003 time frame
    • The program was primarily focused around the “solutions” and products Novell has marketed and sold
    • The overall program responsibility fell to the Enterprise KM group:
          • KM group established the goals, methodologies and processes
          • KM group also identified a set of standard infrastructure tools and supported those
            • No new significant development / implementation was supported
            • This was primarily a case of identifying what was in place and providing an a la carte menu for communities to work with
  • Community Metrics at Novell A brief overview of the program, part 2
    • The CoP program identified levels of communities that largely guided investment and formal support for individual communities
    • There were approximately a dozen “top level” communities over time
    • There were also dozens of informal communities supported to a limited extent
    • The a la carte menu of tools included (among other things):
          • Intranet web sites
          • Mailing lists
          • Enterprise Wiki
          • Team spaces (implemented in LiveLink initially)
  • Setting up my discussion
  • What is a community member? It all started with a simple question
    • The rest of this presentation will focus on a set of metrics we developed at Novell that were manageable, scalable and, I think, provided useful insights
    • To start with, though, early on in our program, we were faced with a simple question from our community leaders:
    • “ How large is my community?”
    • Which quickly turned into:
    • “ What is a community member?”
  • An Answer A definition of community membership
    • Because of our development limitations, we were faced with trying to track membership using either our existing tools or having some type of manual support
    • In looking at our tools, we found that our mailing list infrastructure provided a reasonably good solution:
          • It already had (list) membership management functions
          • People could subscribe / unsubscribe on their own
          • We already knew that the lists needed to be aligned to communities
    • With that insight and no other realistic option, we decided on a definition:
          • You are a member of a community if you are subscribed to any mailing list associated with the community
  • An Implementation How we tracked membership
    • Our mailing list server provided a simple XML format for members for each list
          • We implemented a mechanism to sync this member list data from the XML format into a SQL database
          • This, combined with some integration with other databases already available, provided a wide variety of membership reporting
      • We later added on a mechanism that captured an “event” for each post to a list in the same SQL database, which led to a variety of activity based reporting
  • Membership Metrics
  • Basic Metrics
    • With the basic structure in place, we found we could provide data to answer many questions:
          • How big is any given mailing list?
          • How big is a community?
          • How much change is there in community population over time?
          • Various slicing and dicing by different demographics
    • The following slides provide examples of many of these
  • Basic Metrics Community size
    • Simple to query current size
          • By tracking “join” and “departure” events, it was also straightforward to provide growth over time
    • We could also measure the total overall population of the “community program”
  • Basic Metrics Community penetration
    • Knowing the employee population size (total or by demographics), we could provide measures of “penetration” of the community program as a whole and by community against different groupings
  • Basic Metrics Demographics
    • By combining community membership with HR information, we could provide break downs of membership (both overall program and individual community) on a variety of dimensions
  • Basic Metrics Demographics, continued
  • Basic Metrics Other questions
    • Some other (not necessarily actionable) questions we could easily answer included:
          • How many communities is an employee a member of on average?
          • How many communities is a community member a member of on average?
  • Activity Metrics
  • Activity Metrics
    • Once we implemented a means to capture an “event” for each post in each mailing list, we were able to understand community activity (in this one tool) very easily
    • We also were also able to answer (admittedly, simplistically) another key question we had been asked:
          • “ How many *active* members do I have in my community?”
    • We answered that with:
          • A member is active if they have posted at least one post during the time period of interest.
  • Activity Metrics
    • Some examples of specific metrics we were able to track
  • Activity Metrics
    • We could also gain other potentially useful insights
    • Identifying most active members
          • Potentially useful for identifying SMEs or core team members (or even community leaders)
    • Percent (and number and even individual identification) of lurkers in communities
          • Useful within a community to know who is there but not “active”
          • Useful across communities to know when a community has a significant different rate of “lurkers” compared to other communities
  • Activity Metrics Networks within communities
    • Using the activity data to connect people who corresponded, it was even possible to do data mining to get a sense of the network within a community
  • Performance Management
  • Performance Management
    • Working with the HR group, we eventually were able to community involvement with our performance management program
    • This was achieved by having community involvement embedded in the “employee self service” and “manager self service” portals
  • Performance Management
    • The intent was not that a specific goal was desirable but that this provided a way to initiate conversations between a manager and an employee about involvement, encouraging:
          • Some thought about which communities were valuable for an employee
          • Some reflection on level of involvement and activity
          • Development of employees over time
  • Advanced Metrics
  • Advanced Metrics Some experiments into compound metrics
    • Another simple question prompted some digging into other uses of the data we had:
          • “ Why do we need to provide navigation to community sites [on the intranet], anyway? They don’t get any traffic at all?”
    • The resulting analysis attempts to draw a comparison between “web site visits” and community membership / activity
          • I came to call this measurement “knowledge flow”
    • You can find a detailed description at:
            • http://blog.leeromero.org/2008/11/20/measuring-knowledge-flow-within-a-community-of-practice/
            • The eventual formula I worked out is:
    K c = 2 * P c * A c - P c * A c 2 M c
  • Advanced Metrics Knowledge Flow examples Animations (in four dimensions!) available at: http://blog.leeromero.org/2008/11/21/visualizing-knowledg-flow-in-a-community/
  • Additional Metrics
  • Additional Metrics
    • On top of these metrics based on a mailing list implementation, the Novell CoP program also used a number of additional metrics
    • These included:
          • Web site usage
          • Intellectual asset production
            • Specifically called out white papers as well
          • “ Anecdotes” from community members
          • Knowledge share events
            • And attendance at same
    • For each of these, we provided quarter-to-quarter changes as well
  • Additional Metrics Wiki metrics
    • Another area we investigated but did not lock in on was use / edits of the corporate wiki
    • Novell had (has) a corporate implementation of MediaWiki (in place since about 2003)
    • We explored some metrics related to the Wiki for communities:
          • Establishing a standard Category for each community to use
          • Using that Category assignment, we could:
            • Track usage over time for pages in the category (it was integrated with Omniture)
            • Track edits (“contributions”) over time to pages in that category
            • Potentially identify anyone who edits such a page to be a community member
  • Final Words
  • In Summary
    • Beyond all of the details here, a key takeaway is this:
          • With a very simple definition (of community membership) and a pretty simple technical approach to implement that definition, Novell was able to gain a wide variety of insights
          • Look at the tools you have, make some definitions (even if you know they are not perfect) and start tracking!
    • Also, when you are thinking about what metrics you want to track, make sure all your metrics are actionable
          • How can your metrics be turned into actions to improve your communities?
  • Read more
    • If you would like to read more about the work behind this material, there are extensive write-ups available at:
    • http://blog.leeromero.org
    • Specifically, look under “Collaboration / Communities” on the “Posts by Topic” page