Measuring What MattersDan WeingrodEconsultancy
Harder than ever for businesses to make sense of so much                                  datahttp://www.flickr.com/photos...
Growth in volume of data is outpacing speed at which          companies can make sense of it    | 3 | Online Measurement a...
Do you have a company-wide strategy that ties data        collection and analysis to business objectives?•    Only 22% of ...
Fewer organisations than last year say web analytics driveactionable recommendations that make a differenceDo analytics dr...
62% of companies say that less than half of data is useful  What percentage of the web analytics data you collect is usefu...
To what extent are web analytics integrated into your broader business intelligence strategy?                             ...
Do you integrate web analytics with data from               other sources?             • Yes                           59%...
Main frustrations involved in managing             web analytics      Online Measurement and Strategy Report 2011
What We’ll Talk About Today•   Options for KPI’s•   Getting to Insights•   Testing + Iterating + Learning = Winning•   Int...
If you can’t measure it you can’t manage it…              …but what do we measure, and how?                               ...
Too Much Datahttp://www.flickr.com/photos/mediafury/
Not enough insight
It shouldn’t be that hard…Corporate    Online goalsGoals and        and        KPI’sObjectives    objectives
So what’s the problem? Goals that                                     Goals that  can’t be                                ...
We make it too hard…Corporate    Online goalsGoals and        and        KPI’sObjectives    objectives
Recipe for great KPI’shttp://www.flickr.com/photos/jdickert/
Based on valid datahttp://www.flickr.com/photos/procsilas/
Easy to understandhttp://www.flickr.com/photos/drnewton/
Indicates successhttp://www.flickr.com/photos/8136496@N05/
Or failure
Actionablehttp://www.flickr.com/photos/cottergarage/
The Three Layers of “So What” Test                    Avinash Kaushik• Ask every web metric you report the question "so wh...
These are the top exit pages on our website for the         last month.| 24 |
These are the top exit pages on our website for the         last month.            So what? They dont seem to have changed...
These are the top exit pages on our website for the         last month.            So what? They dont seem to have changed...
These are the top exit pages on our website for the         last month.            So what? They dont seem to have changed...
These are the top exit pages on our website for the         last month.            So what? They dont seem to have changed...
These are the top exit pages on our website for the         last month.            So what? They dont seem to have changed...
Easy to understand                   You, and your boss, can get it    Based on valid data                   No-one can ar...
Pirate Metricshttp://www.flickr.com/photos/72213316@N00/
Pirate Metrics           September 19th - International Talk Like a Pirate dayhttp://www.flickr.com/photos/72213316@N00/
Dave McClure’s: Customer Lifecycle KPI’sAcquisition: users come to the site from various channelsActivation: users enjoy 1...
Pirate Metrics• Acquisition – Did they visit? Did they stay?• Activation – Were they happy? Did they sign up? Did  they in...
Pirate Metrics• Easy to understand?• Based on valid data?• Indicates success…or failure?• Actionable?•
New Product Launch• Acquisition –  – How many visits did we get and were they    greater than baseline?  – Which channels ...
New Product Launch• Activation –  – Did new visitors stay on site longer?  – Did new visitors sign up?  – Did returning vi...
New Product Launch• Retention –  – How many return visitors did we get?  – How many came from marketing or re-    marketin...
New Product Launch• Referral –  – Did they refer others?  – Did their referrers visit and purchase?
New Product Launch• Revenue –  – Did they spend money with us?  – Did they spend money as a new or returning    visitor?  ...
Pirate Metrics• Provides a template for establishment of  simple KPI’s• Based on control and broad knowledge of  channels
siloshttp://www.flickr.com/photos/nakrnsm/
Agency                                                    IT Dept.                                 You    Marketinghttp://...
Recipe for great KPI’s•   Actionable and simple•   Easy to understand•   Based on reality•   Collaborative•   AARRR
Insight• KPI’s are the platform to deliver insights• Not a status quo, instead a status why  and a status where• This shou...
Insight through testing
Testing via constant iteration• Not looking for the answer• Looking for the next step that will lead us  to better results...
Iterative testing Build    Test     Learn
Recipe•   KPI’s•   Hunches•   A/B Testing•   Analytics•   Speed
Example• KPI - Can we increase customer sign up?• Hunch – Maybe it’s the form design• Test – Design two variants of form, ...
Speed•   Build fast•   Fail fast•   Learn fast•   Move to next fast
Iterate• Don’t make wholesale changes• Small changes that can be built upon• Continue to follow hunches• Collaborate with ...
Results• Insights• Regular continuous improvement• Actionable analytics• Better results, lower costs
Measure What Matters• Analytics and Social Media• K.D. Paine
How do you measure relationships?http://www.flickr.com/photos/nataliejohnson/
Some familiar approaches• Put measurement at the beginning• Create a baseline• Define your KPI’s (KBI’s)
Some different•   Define your environment•   Define your investment•   Determine your benchmarks•   Select measurement too...
Social ROI•   Sales or traffic•   Cost savings•   Paid vs earned search rankings•   Cost avoidance•   Social capital
Engagement• Builds a relationship between customers  and brands• Helps promote and protect your brand• Make your products ...
Engagement levels•   Lurking - Impression•   Casual – Like, tweet, click•   Active – Follower, commenter•   Committed – Re...
Engagement levels•   Lurking - A•   Casual - A•   Active - R•   Committed - R•   Loyalist - R           AARRR!
Social Media Analytics•   Measuring engagement•   Similar and different•   Less actionable/less control•   Potential for n...
Summary Insights• KPI’s are the building blocks for real  insights about customers, products and  profits• Iterative testi...
Thank You
Measure what matters
Upcoming SlideShare
Loading in …5
×

Measure what matters

395 views

Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

Measure what matters

  1. 1. Measuring What MattersDan WeingrodEconsultancy
  2. 2. Harder than ever for businesses to make sense of so much datahttp://www.flickr.com/photos/brandnewbrain/
  3. 3. Growth in volume of data is outpacing speed at which companies can make sense of it | 3 | Online Measurement and Strategy Report 2011 |
  4. 4. Do you have a company-wide strategy that ties data collection and analysis to business objectives?• Only 22% of companies have a strategy in place, down from 25% last year.• 78% do not have a strategy, but 63% are working on this.http://www.flickr.com/photos/risager/
  5. 5. Fewer organisations than last year say web analytics driveactionable recommendations that make a differenceDo analytics drive actionable recommendations that make a difference to your organisation? 28% • Yes – definitely 57% • Yes - sometimes 14% • No – not really 1% • Never Online Measurement and Strategy Report 2011
  6. 6. 62% of companies say that less than half of data is useful What percentage of the web analytics data you collect is useful for driving decision-making?2011 27% 35% 28% 10% 0-25% 25-50% 50-75% 75-100%2010 22% 40% 27% 10% Online Measurement and Strategy Report 2011
  7. 7. To what extent are web analytics integrated into your broader business intelligence strategy? 41% 30% 19% 15% 11%Not integrated at all We dont have a We have a common We have common We have common business intelligence technology platform KPIs for web and teams and strategy for web and non-web non-web data processes for web data and non-web data Online Measurement and Strategy Report 2011
  8. 8. Do you integrate web analytics with data from other sources? • Yes 59% • No 41% http://www.flickr.com/photos/carnielewis187 http://www.flickr.com/photos/ghetzu/ Online Measurement and Strategy Report 2011
  9. 9. Main frustrations involved in managing web analytics Online Measurement and Strategy Report 2011
  10. 10. What We’ll Talk About Today• Options for KPI’s• Getting to Insights• Testing + Iterating + Learning = Winning• Intro to Social Media Analytics
  11. 11. If you can’t measure it you can’t manage it… …but what do we measure, and how? | 11 |http://www.flickr.com/photos/iliahi/
  12. 12. Too Much Datahttp://www.flickr.com/photos/mediafury/
  13. 13. Not enough insight
  14. 14. It shouldn’t be that hard…Corporate Online goalsGoals and and KPI’sObjectives objectives
  15. 15. So what’s the problem? Goals that Goals that can’t be can’t be measured measured by online Analytics Goals that are too complex Goals without a clear success metric Goals that wedon’t have data for Goals that Goals have too can’t be many adjusted dependencies
  16. 16. We make it too hard…Corporate Online goalsGoals and and KPI’sObjectives objectives
  17. 17. Recipe for great KPI’shttp://www.flickr.com/photos/jdickert/
  18. 18. Based on valid datahttp://www.flickr.com/photos/procsilas/
  19. 19. Easy to understandhttp://www.flickr.com/photos/drnewton/
  20. 20. Indicates successhttp://www.flickr.com/photos/8136496@N05/
  21. 21. Or failure
  22. 22. Actionablehttp://www.flickr.com/photos/cottergarage/
  23. 23. The Three Layers of “So What” Test Avinash Kaushik• Ask every web metric you report the question "so what" three times.• Each question provides an answer that in turn raises another question (a "so what" again).• If at the third "so what" you dont get a recommendation for an action you should take, you have the wrong metric.• Kill it.
  24. 24. These are the top exit pages on our website for the last month.| 24 |
  25. 25. These are the top exit pages on our website for the last month. So what? They dont seem to have changed in six months| 25 |
  26. 26. These are the top exit pages on our website for the last month. So what? They dont seem to have changed in six months We should focus on these pages because they are major leakage points in our website.| 26 |
  27. 27. These are the top exit pages on our website for the last month. So what? They dont seem to have changed in six months We should focus on these pages because they are major leakage points in our website. So what? We have looked at this report for six months and tried to make fixes, and even after that the pages listed here have not dropped off the report.| 27 |
  28. 28. These are the top exit pages on our website for the last month. So what? They dont seem to have changed in six months We should focus on these pages because they are major leakage points in our website. So what? We have looked at this report for six months and tried to make fixes, and even after that the pages listed here have not dropped off the report. If we can stop visitors from leaving the website, we| 28 | can keep them on our web site.
  29. 29. These are the top exit pages on our website for the last month. So what? They dont seem to have changed in six months We should focus on these pages because they are major leakage points in our website. So what? We have looked at this report for six months and tried to make fixes, and even after that the pages listed here have not dropped off the report. If we can stop visitors from leaving the website, we| 29 | can keep them on our web site. So what? Doesnt everyone have to exit on some page?
  30. 30. Easy to understand You, and your boss, can get it Based on valid data No-one can argue the numbers Indicates success #Winning Actionable You can affect changehttp://www.flickr.com/photos/azugaldia/
  31. 31. Pirate Metricshttp://www.flickr.com/photos/72213316@N00/
  32. 32. Pirate Metrics September 19th - International Talk Like a Pirate dayhttp://www.flickr.com/photos/72213316@N00/
  33. 33. Dave McClure’s: Customer Lifecycle KPI’sAcquisition: users come to the site from various channelsActivation: users enjoy 1st visit: "happy" user experienceRetention: users come back, visit site multiple timesReferral: users like product enough to refer othersRevenue: users conduct some monetization behavior AARRR!
  34. 34. Pirate Metrics• Acquisition – Did they visit? Did they stay?• Activation – Were they happy? Did they sign up? Did they include profile data?• Retention – Did they return? Did they return based on targeted marketing?• Referral – Did they refer others?• Revenue – Did they spend money with us?•
  35. 35. Pirate Metrics• Easy to understand?• Based on valid data?• Indicates success…or failure?• Actionable?•
  36. 36. New Product Launch• Acquisition – – How many visits did we get and were they greater than baseline? – Which channels did visitors come from? – Were visits from existing customers or new customers?
  37. 37. New Product Launch• Activation – – Did new visitors stay on site longer? – Did new visitors sign up? – Did returning visitors, or customers, stay on site longer or take action?
  38. 38. New Product Launch• Retention – – How many return visitors did we get? – How many came from marketing or re- marketing campaigns and which ones?
  39. 39. New Product Launch• Referral – – Did they refer others? – Did their referrers visit and purchase?
  40. 40. New Product Launch• Revenue – – Did they spend money with us? – Did they spend money as a new or returning visitor? – How much did they spend?
  41. 41. Pirate Metrics• Provides a template for establishment of simple KPI’s• Based on control and broad knowledge of channels
  42. 42. siloshttp://www.flickr.com/photos/nakrnsm/
  43. 43. Agency IT Dept. You Marketinghttp://www.flickr.com/photos/nakrnsm/
  44. 44. Recipe for great KPI’s• Actionable and simple• Easy to understand• Based on reality• Collaborative• AARRR
  45. 45. Insight• KPI’s are the platform to deliver insights• Not a status quo, instead a status why and a status where• This should be the hard stuff
  46. 46. Insight through testing
  47. 47. Testing via constant iteration• Not looking for the answer• Looking for the next step that will lead us to better results• Coming up with new answers
  48. 48. Iterative testing Build Test Learn
  49. 49. Recipe• KPI’s• Hunches• A/B Testing• Analytics• Speed
  50. 50. Example• KPI - Can we increase customer sign up?• Hunch – Maybe it’s the form design• Test – Design two variants of form, drive traffic to both• Analytics – Measure results
  51. 51. Speed• Build fast• Fail fast• Learn fast• Move to next fast
  52. 52. Iterate• Don’t make wholesale changes• Small changes that can be built upon• Continue to follow hunches• Collaborate with inbound marketing
  53. 53. Results• Insights• Regular continuous improvement• Actionable analytics• Better results, lower costs
  54. 54. Measure What Matters• Analytics and Social Media• K.D. Paine
  55. 55. How do you measure relationships?http://www.flickr.com/photos/nataliejohnson/
  56. 56. Some familiar approaches• Put measurement at the beginning• Create a baseline• Define your KPI’s (KBI’s)
  57. 57. Some different• Define your environment• Define your investment• Determine your benchmarks• Select measurement tool• Analyze and draw conclusions
  58. 58. Social ROI• Sales or traffic• Cost savings• Paid vs earned search rankings• Cost avoidance• Social capital
  59. 59. Engagement• Builds a relationship between customers and brands• Helps promote and protect your brand• Make your products better
  60. 60. Engagement levels• Lurking - Impression• Casual – Like, tweet, click• Active – Follower, commenter• Committed – Retweet, repost, share, #• Loyalist- Active comments and involvement
  61. 61. Engagement levels• Lurking - A• Casual - A• Active - R• Committed - R• Loyalist - R AARRR!
  62. 62. Social Media Analytics• Measuring engagement• Similar and different• Less actionable/less control• Potential for new insight
  63. 63. Summary Insights• KPI’s are the building blocks for real insights about customers, products and profits• Iterative testing can help us reach insights more quickly• Social analytics are rapidly becoming a part of the essential mix
  64. 64. Thank You

×