Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Harnessing the Power of Product Analytics by Dan Olsen

1,167 views

Published on

I gave this talk on how to use product analytics to optimize your product at Split Software's Decisions Product Management Conference on October 2, 2018.

Published in: Business
  • Be the first to comment

Harnessing the Power of Product Analytics by Dan Olsen

  1. 1. Harnessing the Power of Product Analytics Dan Olsen 10-2-18
  2. 2. n Engineering background: submarine design n Stanford MBA n Product Management leader at Intuit & startups n CEO & Cofounder, TechCrunch winner YourVersion n Product Management consultant n Founder: Lean Product Silicon Valley Twitter: @danolsen My slides: https://dan-olsen.com My Background Copyright © 2018 @danolsen
  3. 3. Product Manager’s motto: Spiderman’s motto: “With great power comes great responsibility” “With great responsibility comes NO power” Copyright © 2018 @danolsen
  4. 4. Product Teams Are Responsible For Product-Market Fit AND Growth Copyright © 2018 @danolsen
  5. 5. n Book giveaway on Twitter n Tweet: include @danolsen n Hashtags #prodmgmt #leanstartup #Decisions2018 @SplitSoftware That’s Why I Wrote Copyright © 2018 @danolsen
  6. 6. Identify highest ROI idea Design and Implement Analyze How the Metric Changes Brainstorm Ideas to Improve Metric Copyright © 2018 @danolsen Lean Product Analytics Process Identify What Your Metrics Are Measure Metrics Baseline Values Evaluate Metrics Upside Potential Global Level Metric Level Select Top Metric Learn & Iterate
  7. 7. At any point, one metric offers the highest return-on-investment (ROI) “Metric that Matters Most” (MTMM) Focus on right metric at right time Copyright © 2018 @danolsen
  8. 8. If you just launched a new product, which would you optimize first? Acquisition: attracting prospects Conversion: turning prospects into customers Retention: ensuring customers remain active Copyright © 2018 @danolsen
  9. 9. If you could track only 1 metric to measure Product-Market Fit, which metric would it be? Copyright © 2018 @danolsen
  10. 10. Retention Rate n Retention rate tracks what % of your customers are still active over time ~80% never use app again Curve either goes to zero or flattens out
  11. 11. Cohort Analysis Copyright © 2018 @danolsen
  12. 12. Improving Retention Rate Over Time= Increasing Product-Market Fit
  13. 13. Improving Retention Rate Over Time= Increasing Product-Market Fit
  14. 14. Improving Retention Rate Over Time= Increasing Product-Market Fit
  15. 15. Unique Visitors in Last Period x Monthly Retention Rate Profit = Revenue - Cost Unique Visitors x Ad Revenue per Visitor Impressions/Visitor x Effective CPM / 1000 Visits/Visitor x Pageviews/Visit x Impressions/PV New Visitors + Returning Visitors Define the Equation of your Business Peeling the Onion Advertising Business Model: Copyright © 2018 @danolsen
  16. 16. View Each Metric as a Gauge Copyright © 2018 @danolsen Minimum Possible Value Maximum Possible Value Current Value
  17. 17. Identifying Your Metric that Matters Most n What is the upside potential of each metric? n How much do we think we can “move the needle”? n What would the revenue impact be? n How many resources will it take to move the needle”? n Developer-days, time, money n Which metric offers best Return on Investment (ROI)? Copyright © 2018 @danolsen
  18. 18. Copyright © 2018 @danolsen Prioritizing Product Ideas by ROI Investment (developer-weeks) Return(ValueCreated) Idea C Idea B Idea D Idea A Idea F 1 1 2 3 4 2 3 4 ?
  19. 19. Types of Metric ROI SituationsReturn Investment Return Investment Return Investment Metric A Good ROI Metric B Bad ROI Metric C Great ROI Copyright © 2018 @danolsen
  20. 20. Case Study Copyright © 2018 @danolsen
  21. 21. • Which metric has highest ROI opportunity? Case Study: Optimizing Friendster’s Viral Loop Active Users Prospective Users Invite Click Succeed Invite click-through rate Conversion rate Don’t Click Fail Invites per sender % of users sending invites • Multiplied together, these metrics determine your viral ratio Users % of users who are active = 15% = 2.3 = 85% Registration Process Copyright © 2018 @danolsen
  22. 22. The Upside Potential of a Metric 0 100% 0 100% 0 ? Registration Process Yield % of users sending invitations Avg # of invites sent per sender 2.3 85% 15% Max possible improvement 0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?% Copyright © 2018 @danolsen
  23. 23. Is anyone feeling a sense of déjà vu right now?
  24. 24. The Upside Potential of a Metric 0 100% 0 100% 0 ? Registration Process Yield % of users sending invitations Avg # of invites sent per sender 2.3 85% 15% Max possible improvement 0.15 / 0.85 = 18% 0.85 / 0.15 = 570% ? / 2.3 = ?% Copyright © 2018 @danolsen Metric B Bad ROI Metric A Good ROI Metric C Great ROI
  25. 25. Okay, so how can we improve the metric? n How do we increase the average number of invites being sent out per sender? n For each idea: n What’s the expected benefit? (how much will it improve the metric?) n What’s the expected cost? (how many engineer- days will it take?) n You want to identify highest ROI idea Copyright © 2018 @danolsen
  26. 26. After Launching Address Book Importer… Copyright © 2018 @danolsen
  27. 27. After Launching Address Book Importer… Copyright © 2018 @danolsen
  28. 28. After Launching Address Book Importer… Copyright © 2018 @danolsen
  29. 29. Identify highest ROI idea Design and Implement Analyze How the Metric Changes Brainstorm Ideas to Improve Metric Copyright © 2018 @danolsen Lean Product Analytics Process Identify What Your Metrics Are Measure Metrics Baseline Values Evaluate Metrics Upside Potential Metric Level Select Top Metric Learn & Iterate Global Level
  30. 30. Upping Your Experimentation Game n Simultaneous tests vs. before-and-after n Rollout to any % of users vs. only 100% Feature flags n Testing multiple variants vs. 1 alternative n Experimentation platform: buy vs. build n Experimentation velocity learning velocity Copyright © 2018 @danolsen
  31. 31. I’m teaching a Lean Product Management Workshop in SF Oct 24th San Francisco https://danolsen.eventbrite.com Save 15% with code SPLIT
  32. 32. Thanks! @danolsen https://dan-olsen.com Will notify book winner on Twitter Happy to connect on LinkedIn

×