Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

UCO16 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

111 views

Published on

Unity Connect Online - June 20 - 23, 2016

Published in: Software
  • Be the first to comment

  • Be the first to like this

UCO16 - An Independent Evaluation of Third-Party SharePoint Analytics Offerings

  1. 1. Unity Connect Online 2016 #UCO16 unityconnect.com An Independent Evaluation of Third-Party SharePoint Analytics Offerings Jonathan Ralton All trademarks and registered trademarks are the property of their respective owners.
  2. 2. Jonathan Ralton BlueMetal An Independent Evaluation of Third- Party SharePoint Analytics Offerings
  3. 3. Agenda Setting the Stage Client Story Measuring and Trending Definitions Framework Examples SharePoint Analytics Third-Party Tool Candidates Which would you choose? Epilogue Which did I choose? How does the story end? Wrapping Up Questions
  4. 4. Setting the Stage An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  5. 5. Presenter Jonathan Ralton • Senior Information Architect • SharePoint IT Pro since 2005 (WSS/SPS) • No coding! • Document Management, Content Management, Knowledge Management… @jonralton jonathanr@bluemetal.com blog.jonralton.net
  6. 6. Client Story An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  7. 7. Overview • Food and Beverage Manufacturing Company • One of the top industry leaders in the country • One of the top industry leaders in the world • Employees in dozens of countries • Hundreds of household brand names • Some with $1b market shares
  8. 8. Intranet • Redesigned by BlueMetal and relaunched in late 2014 • Heavy on video content • Social focus • Personalization
  9. 9. Challenges Google External Media Outlets Chatter/ Rumor Mill Emails Bing
  10. 10. Challenges Communication Justification • Need to prepare for new management oversight • Need to publish appropriate news at appropriate times • Want to maximize investment in the new facility of the intranet
  11. 11. Objectives Content and Visitors • Learn what content is being consumed and by whom • Learn which types of content are most engaging • Gain better insight into what kinds of content employees will respond well to, and in turn continue to return to myCorptown for Employee Communications • Help to justify team’s value in managing myCorptown • Demonstrate higher employee satisfaction in being informed • Demonstrate higher employee engagement
  12. 12. Objectives Adoption Engagement
  13. 13. Visitors/Visits/Constituencies New Employees New Users Lapsed Users Executive Employees Mobile Users Tablet Users Laptop Users Desktop Users Kiosk Users Hourly Employees Salaried Employees Departments Off-Hours Work Hours Break Time
  14. 14. Current Tool
  15. 15. Measuring and Trending Must Love Term Sets
  16. 16. WEB ANALYTICS def·i·ni·tion [dèffə nísh'n]
  17. 17. Web Analytics Web analytics is the measurement, collection, analysis, and reporting of web data for purposes of understanding and optimizing web usage.
  18. 18. METRIC def·i·ni·tion [dèffə nísh'n]
  19. 19. Metric A metric is a something that can be measured based on quantifiable data, or at the very least, estimated based on anecdotal data.
  20. 20. Metrics w/Tools Web analytics tools mainly provide metrics for consumption via reports and APIs.
  21. 21. KEY PERFORMANCE INDICATOR (KPI) def·i·ni·tion [dèffə nísh'n]
  22. 22. Key Performance Indicator A key performance indicator (KPI) is a visual indicator that tells a story about one or more measurements, in context.
  23. 23. Key Performance Indicators w/Tools Web analytics tools may already provide some KPIs in dashboards.
  24. 24. Metrics and KPI Framework Goals Signals Metrics KPIs
  25. 25. Goals Adoption Engagement RetentionTask Success Happiness
  26. 26. Sample Goals – Adoption Ensure new employees and new users are engaging with myCorptown content for the first time, including the email newsletter Build employee buy-in by telling our story through our vehicles Reach our employees ‘where they are’ including via mobile and kiosks ‘Give them what they want’ by being able to respond to popularity of content Get our employees to think of going to myCorptown first
  27. 27. Sample Goals – Engagement Get our employees to give feedback on content including likes and comments, submitting questions, etc. Have our employees be conversant in an informed manner about company news with fellow colleagues Get our executive team to choose myCorptown as their communication vehicle Get our employees to help each other (Employees as Communicators)
  28. 28. Sample Goals – Retention Get our users to return to myCorptown more often (more than 1x per week) Encourage our lapsed users to return to myCorptown Reduce stale content in areas on myCorptown where dynamic content is expected by our users
  29. 29. Sample Goals – Task Success ‘Give them what they need’ in addition to what our users want, including via search Solve our users’ problems via self-service where possible (without a help desk call)
  30. 30. Sample Goals – Happiness Ensure our employees are able to articulate the company story • What we stand for • How does my work fit into the story • How does my work touch consumers Support a more transparent culture at the company Decrease frustration during activities on myCorptown Support our employees through times of change
  31. 31. Sample Signals Goal Get our users to return to myCorptown more often (more than 1x per week) Signals • Providing content relevant to users • Identify users who return to visit the site within a week period
  32. 32. Sample Signals Goal Get our employees to give feedback on content including likes and comments, submitting questions, etc. Signals • Identifying posts that have higher numbers of views • Identifying posts that have high numbers of comments • Monitoring submitted questions
  33. 33. Sample Signals Goal ‘Give them what they need’ in addition to what our users want, including via search Signals • Identifying usage of tools and links that have typically been difficult to find • Monitoring search results for successful and failed queries
  34. 34. Sample KPI w/Trending #1 • Number of retained user home page views minus bounces as percentage of all views • Over time period of 7 days • Indication of current metric against target w/in a certain window • Trend 57% Stickiness Home Page Target 70% Window 7 Days
  35. 35. Sample KPI w/Trending #2 • Number of views during off- work hour segments • Over time period of 5 days • Indication of current metric against target w/in a certain window • Trend 1.6k Time-Based Visits Off-Work Target 2k Window 5 Days
  36. 36. Sample KPI w/Trending and History #1 • Number of posts related to influencers • Over time period of 30 days • Indication of current metric against target w/in a certain window • Performance against last window and previous year’s window • Example of where higher metric is better 38 Influencers Posts Target 50 Window 30 Days YoY +70.1% Last Month -23.9%
  37. 37. Sample KPI w/Trending and History #2 • Number of questions submitted to portal team • Over time period of 30 days • Indication of current metric against target w/in a certain window • Performance against last window and previous year’s window • Example of where lower metric is better 12 Portal Questions Submitted Target 15 Window 30 Days YoY -156.5% Last Month -10.4%
  38. 38. Sample KPI w/Snapshot #1 Average Length of Visit > 5 m > 4 m > 3 m > 2 m > 1 m • Average visit length • Segmented into groups • Current metric for one window • Shows spread/breakdown detail • Does not show trend
  39. 39. Sample KPI w/Snapshot #1 • Average visit length in minutes • Over time period of 7 days • Indication of current metric against target w/in a certain window • Trend 1.8 Length of Visit Home Page Target 2.0 Window 7 Days
  40. 40. Sample KPI w/Snapshot #2 • Rating of frustration level on portal from survey • Current metric for one window • Shows spread/breakdown detail • Does not show trend 0 1000 2000 3000 4000 5000 6000 Low Medium High Frustration Level Frustration Level
  41. 41. Sample KPI w/Snapshot #2 • Most common rating of frustration level from survey • Over time period of 1 fiscal quarter • Indication of current metric against target w/in a certain window • Performance against last window and previous year’s window • Example of where metric is abstracted M Portal Frustration Level Target M Window Quarter YoY +27.6% Last Quarter +5.8%
  42. 42. SharePoint Analytics An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  43. 43. Changes ⓮ → ⓯
  44. 44. Changes ⓮ → ⓯ • Web Analytics Web Part is not supported anymore • Reports Removed • Browser Traffic • Top Users • Referring URL
  45. 45. Changes ⓮ → ⓯ • Engine finds relevant information based on clicks, views, etc. • There are now visual indicators and usage numbers based on number of views and number of unique visitors • Clicks and views are counted for each document • Content can be recommended • Search results influenced by the priority of an item • Ability to sort search results by "hit" • Engine is now extensible for third-party solutions
  46. 46. SharePoint Usage Reporting Usage This report shows historical usage information about the site collection/site, such as the number of views and unique users. Use this report to identify usage trends and to determine times of high and low activity.
  47. 47. SharePoint Search Reporting Number of Queries This report shows the number of search queries performed. Use this report to identify search query volume trends and to determine times of high and low search activity. Top Queries by Day/Month This report shows the most popular search queries. Use this report to understand what types of information visitors are seeking. Abandoned Queries by Day/Month This report shows popular search queries that received low click-through. Use this report to identify search queries that might create user dissatisfaction and to improve the discoverability of content. Then, consider using query rules to improve the query's results. No Result Queries by Day/Month This report shows popular search queries that returned no results. Use this report to identify search queries that might create user dissatisfaction and to improve the discoverability of content. Then, consider using query rules to improve the query's results. Query Rule Usage by Day/Month This report shows how often query rules trigger, how many dictionary terms they use, and how often users click their promoted results. Use this report to see how useful your query rules and promoted results are to users.
  48. 48. SharePoint Library Reporting Most Views Most Views by Unique Users Most Recommendation Clicks
  49. 49. Third-Party Tool Candidates An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  50. 50. Requirements – Compatibility A tool for myCorptown analytics will require compatibility with (or extensibility to support) traffic delivered through…
  51. 51. Requirements – Awareness In order to provide the most visibility into activity on myCorptown, a tool for analytics should be specifically aware of traffic, content, and actions related to… • Social Intranet • Video Cloud
  52. 52. Requirements – Content Segmentation Intranet administrators would like to segment analytics data by the types of content consumed. This would require support for… • Metadata • File Extension
  53. 53. Requirements – Visitor Segmentation Along with the desire to segment the content consumed within myCorptown, intranet administrators will need to examine various properties of the visits and visitor as well. These attributes of a visit or visitor should be supported… • Platform • Mobile Device • Browser • Date and Time • Session • Microsoft SharePoint 2013 • User Profile Property • User Audience Membership • Microsoft Active Directory • User • User Attribute • User Group Membership
  54. 54. Requirements – Extensibility Reusable Metrics In order for a dashboard or other display and recalculation of raw metric data into KPIs and reports, an API or REST interface should ideally be provided by the tool. Email Campaigns In addition to native web traffic on myCorptown, the tool should support UTM tags commonly used in conjunction with email campaigns and the hyperlinks that refer visitors to the site.
  55. 55. Requirements – Pricing For the purposes of evaluation, the following data points were used: • Page Views ~1,000,000/month • Users ~30,000
  56. 56. Google Analytics Google Analytics helps analyze visitor traffic and paint a picture of your audience and their needs. Mainly designed for e-commerce websites, it is a cloud-only/SaaS offering.
  57. 57. Google Analytics Trackable Attributes/Actions • Page Views • Clickpaths • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Tracking occurs via JavaScript- based page tags
  58. 58. Google Analytics • Data is often based on a subset, or percentage of sessions • This is due to the practice of reporting trends in the subset to reduce processing time • This can occur in the reports, during the data collection itself, or in both places
  59. 59. Google Analytics Key Benefits Key Drawbacks  In-Page Analytics  Clickpath Visualization  Sparkline KPIs  Real-time  Cloud-based  Google takes ownership of your metrics data and may use it how they see fit  Not ‘SharePoint-aware’  Does not analyze search queries  No video metrics  No Yammer integration  No Sitrion integration  Visitor attributes require customization  Shared login/Google account required
  60. 60. ActualMetrics Angelfish With its roots in Urchin, Angelfish analyzes IIS logs as a completely separate application, usually installed on a server apart from SharePoint.
  61. 61. ActualMetrics Angelfish Trackable Attributes/Actions • Page Views • Clickpaths • File Downloads • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Angelfish relies on the IIS logs • Although tagging with JavaScript is supported, it is not required
  62. 62. ActualMetrics Angelfish • This tool appears to be in its infancy at this point in time • A lot of features are in the works/slated for a future release • The audience for this tool seems to be more technical users than business users • The interface seems like it would be more difficult to navigate for someone outside of IT
  63. 63. ActualMetrics Angelfish Key Benefits Key Drawbacks  Spike analysis—second-level time segmentation  Unlimited nested segments  Bandwidth analysis  Broken link analysis  Six different tracking methods for visitors  Not ‘SharePoint-aware’  Does not analyze search queries  No video metrics  No Yammer integration  No Sitrion integration  Visitor attributes require customization  Reports are only exportable in unformatted form (CSV, XML)  Separate hardware recommended
  64. 64. Adobe Analytics Formerly known under the brand Omniture, Adobe purchased this analytics suite and it is now part of their Marketing Cloud offering. Mainly designed for e-commerce websites, it is a cloud-only/SaaS offering.
  65. 65. Adobe Analytics Trackable Attributes/Actions • Page Views • Clickpaths • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Tracking occurs real-time via JavaScript-based page tags
  66. 66. Adobe Analytics • Adobe has put a lot of effort into the user interface to enable visualization of data
  67. 67. Adobe Analytics Key Benefits Key Drawbacks  Sparkline KPIs  Real-time  Cloud-based; low overhead  Not ‘SharePoint-aware’  Does not analyze search queries  No video metrics  No Yammer integration  No Sitrion integration  Visitor attributes require customization
  68. 68. HarePoint Analytics for SharePoint Using its own analytics engine separate from SharePoint and any server logs, HarePoint Analytics installs as part of SharePoint while using its own SQL databases.
  69. 69. HarePoint Analytics for SharePoint Trackable Attributes/Actions • Page Views • Clickpaths • File Downloads • Search Queries • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Not requiring JavaScript tags, the tool utilizes its own tracking within SharePoint to provide metrics
  70. 70. HarePoint Analytics for SharePoint • HarePoint seems committed to this product, as it has adapted it from SharePoint 2007 through 2010 and 2013, however they have yet to integrate social metrics
  71. 71. HarePoint Analytics for SharePoint Key Benefits Key Drawbacks  Workflows tracked  Web part for analytics display within SharePoint  Aware of content within SharePoint, not just actions by visitors o Unused content can be identified  Real-time  No additional hardware required  Must access reports from Central Administration  Can’t track external links  No social tracking
  72. 72. Intlock CardioLog Analytics Designed specifically for SharePoint and covers all aspects of usage including portals, extranets, collaboration sites, and social. It is installed on a separate SQL server.
  73. 73. Intlock CardioLog Analytics Trackable Attributes/Actions • Page Views • Clickpaths • File Downloads • Search Queries • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Tracking occurs real-time via JavaScript-based page tags • IIS logs can be imported to integrate historic data • It does not rely on SharePoint’s built-in analytics or logs
  74. 74. Intlock CardioLog Analytics • This tool has great in-page feedback options including: • Message bar • Survey • A/B multivariate testing
  75. 75. Intlock CardioLog Analytics Key Benefits Key Drawbacks  Custom dashboards  Direct integration with Active Directory and SharePoint user profiles  Display within SharePoint  Page metadata  Automatically generated reports  Identify unused content  Sitrion integration  Requires separate hardware
  76. 76. Piwik SharePoint Analytics With a highly evolved UI for reports, Piwik offers solid SharePoint analytics by activating a feature per site collection.
  77. 77. Piwik SharePoint Analytics Trackable Attributes/Actions • Page Views • Clickpaths • File Downloads • Search Queries • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Tracking occurs via JavaScript- based page tags
  78. 78. Piwik SharePoint Analytics • The installation of Piwik is very streamlined • A pre-configured server image is delivered and the tool automatically populates the tracking code into all relevant areas
  79. 79. Piwik SharePoint Analytics Key Benefits Key Drawbacks  Real-time viewing of visitor actions  Sparkline KPIs  Ease of setup  Lack of social tracking  Separate hardware required
  80. 80. WebTrends SharePoint Analytics Closely aligned with Microsoft, Webtrends is a marquee vendor for web analytics.
  81. 81. Piwik SharePoint Analytics Trackable Attributes/Actions • Page Views • Clickpaths • File Downloads • Search Queries • Visitors • Browser/Platform/Device • Date and Time Analysis Method • Embedded JavaScript tags in the page send information (encrypted in transit) to the tool
  82. 82. Webtrends SharePoint Analytics • Webtrends seems to have a particular focus on adoption and engagement. • Their pitch is “consumable and actionable insights for business users” • It has a well-polished UI
  83. 83. Webtrends SharePoint Analytics Key Benefits Key Drawbacks  Cloud-based; no overhead  Targeted for consumption by business users  Plug-ins for Brightcove  Sitrion integration  Feature-heavy product; only small subset of feature set is required
  84. 84. Which would you choose? An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  85. 85. Epilogue An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  86. 86. Comparison Matrix
  87. 87. Areas of Comparison • Compatibility • Tracking Method • On-Premise vs. SaaS • Content and Process Awareness • Content Segmentation • Visits/Visitor Segmentation • Reporting • Feedback • Extensibility • Price
  88. 88. Which did I choose? An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  89. 89. Recommended Tool
  90. 90. Why? • This recommendation is based largely on the feature set offered by CardioLog • How those features align with the needs of the client in tracking activity on myCorptown • Content as well as activity is monitored • Lack of action can be tracked on documents and items • In-page feedback mechanisms were very attractive
  91. 91. Why? • Sitrion analytics are built-in and will require little to no customization • Brightcove analytics should be able to be integrated via their API and brought into CardioLog
  92. 92. Why? • Other tools… • Did not have as extensive a feature set to support the client’s requirements • Were less SharePoint-specific • Did not allow for business users to consume their reports as easily • Cost too much
  93. 93. How does the story end? An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  94. 94. Chosen Tool
  95. 95. …and they lived happily ever after. • Decided to address some of their near-term goals • Have reports against them based on certain metrics • Some of the KPIs would be able to be developed from those metrics (manually) • In order for this to take place, an augmentation of current information being sent over the Internet to Google Analytics is required • This needs to include sending over key user attributes as well as some classification of the pages being viewed • Integration with SharePoint user profile data will be required to track things such as a user’s department
  96. 96. Warnings • Complete user information such as first and last names should not be sent over to Google Analytics, as the data may be reused by Google in any manner • Steps should be taken so that any confidential information or trade secrets are not accidentally injected into the Google ecosystem in the form of metatags, page titles, etc.
  97. 97. Wrapping Up An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  98. 98. Wrapping Up The views expressed in this presentation are not in any way endorsed by any of the product vendors, and I warrant no accuracy in the data presented.
  99. 99. Wrapping Up Although I concluded that CardioLog Analytics is the leading candidate for this particular engagement, this does not by any means signify future disqualification by myself or BlueMetal in future evaluations of the other tool candidates.
  100. 100. Wrapping Up In fact, another tool might be the best we could recommend to a completely different future client depending on their needs.
  101. 101. Questions An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  102. 102. Reference An Independent Evaluation of Third-Party SharePoint Analytics Offerings
  103. 103. Resources Tool Website Tool Download/Request Tool Demo ActualMetrics Angelfish Link Adobe Analytics Managed Metadata Overview Link Google Analytics HarePoint Analytics for SharePoint Link Link Intlock CardioLog Analytics Link Piwik SharePoint Analytics Webtrends SharePoint Analytics Google Analytics for SharePoint 2013/Office 365 (CodePlex)

×