Lies, Damned Lies & Vanity Metrics

3,114 views

Published on

The slides on a session which I conducted to the startup teams in the technology startup incubator JFDI.Asia. The session which I conducted are broken into 4 parts: managing user growth, vanity and growth metrics, funnelling and usability testing.

Published in: Technology, Business

Lies, Damned Lies & Vanity Metrics

  1. 1. Lies, Damned Lies &Vanity MetricsBernard LeongA Simple Guide to Feedback, Analytics &Iteration in Product Development1
  2. 2. “There are three kinds of lies:lies, damned lies & statistics.”- Mark Twain2
  3. 3. A good theoretical physicist is one who makesall the mistakes as quickly as possible.entrepreneur3
  4. 4. 4
  5. 5. Feedback Generation• How do we collect and generate user feedback?• Qualitative: Surveys, Talk to customers, Createchannels via social media (twitter/facebook).• Quantitative: Google Analytics, Mixpanelmetrics.• User interface & experience testing: Usereducation and perception of how productworks.5
  6. 6. • Qualitative - Break it into the following buckets:• Spam/Troll & Insults - Ignore• “Blue Sky” or “N=1” comments in the form of“you should build this or that” - Think about itbut don’t act• “N>50” comments on a feature or usersdemanding for a feature - Ascertain whetherpeople really want it and then act.Feedback MaintenanceObviously, you need to find metrics to do that ...6
  7. 7. Avoid Vanity Metrics• Number of hits.• Number of page views.• Number of visits.• Number of unique visitors.• Number of followers/friends/likes• Time on site/number of pages.• Emails collected• Number of downloads7
  8. 8. Framework for User Growth8
  9. 9. Element Function Relevant MetricsAcquisitionGenerate attention throughvarious meansTraffic, Mentions, Cost per Clicks,Search Results, Cost of acquisition,Open RateActivationTurn the resulting drive-byvisitors into users who areenrolledEnrollments, signups, completedonboarding process, used the serviceat least once, subscriptionsRetentionConvince users to returnrepeatedly or exhibitstickiness behaviorEngagement, time since last visit, daily& monthly active use, churns.RevenueBusiness outcomesdepending on business modelCustomer lifetime value, conversionrate, shopping cart size & click thrurevenueReferralViral or word of mouthinvitations to other usersInvites sent, viral coefficient, viral cycletime.9
  10. 10. How to identify key metricsAcquisitionActivationRetentionRevenueReferralHow do users discover you?Do users have a frictionless orseamless first experience?Do users return?Do users want to pay youmoney?Do users spread the word?10
  11. 11. Constructing a FunnelInternet Marketing - Google Ads200K impressions = 20K click thrus(10%)User Signups20K click thrus = 2K signups (10%)User create activity2K signups = 200 users returnwith 5 actions (10%)User pay for premiumservice = 20 usersUsers sendinvites to otherusers = 2 usersHow do users discover you?Do users have a frictionless orseamless first experience?Do users return?Do users want to pay youmoney?Do users spread the word?11
  12. 12. Source: Lean AnalyticsUsability Test12
  13. 13. Figuring out the customer lifecycleand working out the metrics whichyou want to measure.Case Study: SaaSservices13
  14. 14. References• Alistair Croll & Benjamin Yoskovitz, “LeanAnalytics”• Jon Yongfook, “21 Actionable Growth HackingTactics”, http://yongfook.com/actionable-growth-hacking-tactics.html• Ash Maurya: 3 Rules to actionable metrics in alean startup: http://www.ashmaurya.com/2010/07/3-rules-to-actionable-metrics/14

×