Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Building Products Quantitatively

1,297 views

Published on

Presentation for the Founder Institute mentoring session.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

Building Products Quantitatively

  1. 1. Building Products Quantitatively in SaaS Businesses Soren Harner, Bigcommerce © 2003 – 2013 Bigcommerce Pty. Ltd. Wednesday, 6 November 13
  2. 2. About me 2012 VP Engineering Stage 2 Startup Venture-backed $35MM 2010 Founder & Datavore Stage 1 Startup Bookstrapped 2007 VP Engineering Stage 2 Startup 2000 Search & Analytics, Stage 2 Startup Stage 3 Acquisition Venture-backed > $80MM Director Engineering Wednesday, 6 November 13
  3. 3. Hill Climbing Wednesday, 6 November 13
  4. 4. SaaS Business Levers • • • • • • Cost to acquire customers (CAC) Average Revenue per User (ARPU) Churn • Goal: Negative revenue retention (upgrades) Conversion (CVN) by channel • Payback Life time value (LTV) • • LTV = ARPU/Churn LTV / CAC > 1 is success! Monthly Recurring Revenue (MRR) • Magic Number Wednesday, 6 November 13 http://blog.asmartbear.com/coc-cancellation-saas-metric.html
  5. 5. LTV / CAC Model “successful” 1.2 0.9 0.6 0.3 0 Q1-10 Q2-10 Q3-10 Q4-10 Q1-11 Q2-11 Q3-11 Q4-11 To IPO range is maybe 4 to 6 LTV / CAC today Fine print: Like all metrics use to track, not as strategy Wednesday, 6 November 13
  6. 6. Tested hundreds of versions Wednesday, 6 November 13
  7. 7. Signup, upgrade, onboarding • Invest in a great experience • Test with usertesting.com • Track the funnel: • Number of trials • Leads to Trials to Paid by weekly cohort • Onboarding success Wednesday, 6 November 13
  8. 8. Product UX Testing Numbers don’t tell you how to improve. Workshops (personas) Concepting (sketches) Prototypes Qualitative (Axure) You can use an iPad at a Cafe and show keynote. Key point is to test mockups on people first. Wednesday, 6 November 13 Quantitative (Loop11)
  9. 9. Feedback Wednesday, 6 November 13
  10. 10. Measure Product Success • Customer success • • • • • Wednesday, 6 November 13 Time to first sale Time to launch Trend monthly or weekly cohorts Engagement score during trial Churn and conversion predictive models
  11. 11. Feature KPI • Before you build a feature, define a success KPI • • Wednesday, 6 November 13 e.g. Number of product reviews completed e.g. 30% fewer abandoned carts • e.g. completing task more quickly • Add instrumentation before and after • Deploy to subset of customers to test • Consider Mixpanel
  12. 12. Split Test Important Things • Persona analysis and customer interviews on new brand • Route traffic p new, 1-p old (control) • Measure effect on conversion with 95% confidence interval of a difference and ZScore Wednesday, 6 November 13
  13. 13. A/B Testing Report and significance p +/- (1.65 x SE) https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html Wednesday, 6 November 13
  14. 14. Network Diffusion Atlassian JIRA to OS communities. SEO / SEM limitations — budget and effectiveness. Wednesday, 6 November 13
  15. 15. Power law Think 80/20 rule. Store revenue, traffic, disk usage, number products, etc. http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html Wednesday, 6 November 13
  16. 16. Continuous Delivery • Feature flag everything and gently deploy • Deploy daily — huge difference! • Read 12factor.net (dev, staging, production) • Monitor (i.e. NewRelic) • Practice BDD & continuous integration • Always have a recovery plan Wednesday, 6 November 13
  17. 17. Ask permission, get feedback Wednesday, 6 November 13
  18. 18. User Stories Wednesday, 6 November 13
  19. 19. Commutative flow Wednesday, 6 November 13
  20. 20. Startup Friendly Stack • Use Ruby Sinatra (or Clojure) + CoffeeScript + Backbone + Handlebars on Heroku • Nesta for CMS and learn textmate, Haml & Sass, see https://github.com/ sharner/Workshop-Site • HTML5 mobile first, responsive design • Use Mailchimp, Wufoo, SurveyMonkey • R for data (http://r-project.org) — treat as code, put in Github • Log events to understand customer success (see Heroku Plugins) Wednesday, 6 November 13
  21. 21. Peopleware • Do learn to code simple website changes and growth hacking • Learn some stats • Do find a technical co-founder • Don’t offshore • Don’t churn people • Do find designer who codes frontend and does user testing Wednesday, 6 November 13

×