1. Measuring Org
Engagement
Bridging the organizational Intranet needs and ‘social’
By: Ronald Sunarno

2. Employee Engagement is a growing topic

3. A framework for measurement:
Organization Engagement Quotient
Highly engaged organization:
1. Productivity
2. Employee satisfaction
3. Knowledge Base
4. Level of collaboration
5. Expertise sharing

4. A framework for measurement
Highly engaged organization:
1. Productivity – measured from operational and organizational KPI
2. Employee satisfaction – measured from survey, usage statistics
3. Knowledge Base
4. Level of collaboration
5. Expertise sharing

5. A framework for measurement
Highly engaged organization:
1. Productivity
2. Employee satisfaction
3. Knowledge Base
4. Level of collaboration
5. Expertise sharing

6. Measuring Success on Knowledge Base
Currently Accessible Out-of-the-box metrics from typical Intranet platforms
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
• Number of usage (clicked, views)
• Number of queries
• Number of abandoned queries

7. Measuring Success on Knowledge Base
Out-of-the-box metrics
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
• Number of usage (clicked, views)
• Number of queries
• Number of abandoned queries
Shows an indication of “attempt” but not necessarily success
No dimension of time – how long to get to the answer?

8. Measuring Success on Knowledge Base
Out-of-the-box metrics
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
• Number of usage (clicked, views)
• Number of queries
• Number of abandoned queries
“Likes and rating” are explicit actions from user
Potentially stronger signal but how many took the time?

9. Measuring Success on Knowledge Base
Out-of-the-box metrics
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
Shows quantity of resources, but what about quality?
Or relevance? – are the quality resources actually needed?
• Number of usage (clicked, views)
• Number of queries
• Number of abandoned queries

10. Measuring Success on Knowledge Base
Out-of-the-box metrics
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
• Number of usage (clicked, views)
• Number of queries
We know people are looking for X, but did they find it in the end?
• Number of abandoned queries

11. Measuring Success on Knowledge Base
Out-of-the-box metrics
• Number of visits
• Number of discussions in forum
• Number of likes
• Content rating
• Number of documents stored
• Number of usage (clicked, views)
So close!!! But to measure success, I don’t care how many
• Number of queries
people are abandoning their search for the term “TPS form”.
• Number of abandoned queries
I care on how many people are giving up for any term

12. out-of-the-box analytics is:
Focused on the statistic of the
resources, Not organizational
insight.
Useful for content champions,
but useless for project sponsor.

13. Knowledge Base in action
People are used to Google. They search

14. Knowledge Base in action
They go through the search result, click on a few things
Maybe try an analogous term, then click a few more

15. Knowledge Base in action
If they’re lucky, they’ll eventually find something and stop looking

16. Knowledge Base in action
“Time to Knowledge”:
The time it takes for someone to find a resource until they stopped looking.
It is indicative of a successful search. Or if it’s too long – a failed search

17. Time-to-Knowledge
Low avg shows:
effective use of the Intranet to share
high quality & highly relevant
resources
High avg shows:
most people cant find what
they want
SUCCESSFUL KNOWLEDGE BASE
FAILED KNOWLEDGE BASE
Intranet ROI can easily be calculated in how an Intranet improve the avg. time-to-knowledge
Opportunity! All these things are not available out-of-the box

18. Collaboration in action
Ron found Jan’s document
through search
Andrew found’s Ron work…
cycle continues

19. Collaboration in action
He leverage Jan’s work and
created a derivative work in
half the time
Andrew found’s Ron work…
cycle continues

20. Collaboration in action
Ron’s work is reviewed by his
peers. The quality increases
Andrew found’s Ron work…
cycle continues

21. Measuring Collaboration
Avg. no. of contributors per document
Shows:
- People are not playing document ping
pong via email
- People are actually not working in silo
Opportunity! All these things are not available out-of-the box

22. Measuring Collaboration
• Avg. no. of crossteams/deparments/offices
contributions
• No. of files available for viewing outside
their departments
Shows: effective knowledge sharing
across functions/locations, etc.
Opportunity! All these things are not available out-of-the box

23. Who’s the influencer?
• Top people whose documents been
copied the most!
Opportunity! All these things are not available out-of-the box

24. Empower the users
• Is MY ORGANIZATION working together?
• Is MY DEPARTMENT working together
• Is MY TEAM working together?
• Who have viewed and read MY DOCUMENTS?
• How many of MY DOCUMENTS have been leveraged?

25. Innovation Engine
• Trending “topic”
people are writing documents about social organization in the Intranet,
should we as a company look into what it’s all about
Opportunity! All these things are not available out-of-the box

26. Summary #1
• Organization Engagement Quotient is a key set of metrics useful to
CIOs and COOs in having a birds eye view of:
• The organizational culture (are people working together?), and
• efficiency (the pace in which people are finding organizational information)
• Multiple companies are looking at the problem (Microsoft,
WebTrends, etc.), however there’s still a huge gap in the market.

27. Summary #2
• Looking at Intranet as a collaboration platform from the angle of “EQ”
gives a glimpse of
a working ideal model for Intranet as an
enabler for creating Social Organization

28. New revelations when
looking at Intranet as a
Social Platform

29. Collaboration Issue #1: Lack of social
credence
“Jan didn’t get the credit of Andrew’s work.”
Wouldn’t it be nice if Jan is notified if anyone have read, updated,
copied his work?
Opportunity! All these things are not available out-of-the box

30. Collaboration Issue #2: Virtual water
cooler
“Andrew didn’t know Ron is working on the same thing as what he’s
doing”
Big shift: tearing down the walls. Automatically announce what people
are doing
With privacy settings
Opportunity! All these things are not available out-of-the box