Building Products
Quantitatively in SaaS
Businesses
Soren Harner, Bigcommerce

© 2003 – 2013 Bigcommerce Pty. Ltd.

Wednes...
About me
2012 VP Engineering

Stage 2 Startup
Venture-backed
$35MM

2010 Founder & Datavore

Stage 1 Startup
Bookstrapped
...
Hill Climbing

Wednesday, 6 November 13
SaaS Business Levers
•
•
•
•
•
•

Cost to acquire customers (CAC)
Average Revenue per User (ARPU)
Churn

•

Goal: Negative...
LTV / CAC

Model
“successful”

1.2
0.9
0.6
0.3
0
Q1-10

Q2-10

Q3-10

Q4-10

Q1-11

Q2-11

Q3-11

Q4-11

To IPO range is m...
Tested hundreds of versions
Wednesday, 6 November 13
Signup, upgrade, onboarding
• Invest in a great experience
• Test with usertesting.com
• Track the funnel:
• Number of tri...
Product UX Testing
Numbers don’t tell you how to improve.

Workshops
(personas)

Concepting
(sketches)

Prototypes

Qualit...
Feedback

Wednesday, 6 November 13
Measure Product Success
•

Customer success

•
•
•
•
•
Wednesday, 6 November 13

Time to first sale
Time to launch

Trend m...
Feature KPI
•

Before you build a
feature, define a success
KPI

•
•

Wednesday, 6 November 13

e.g. Number of
product revi...
Split Test Important Things
• Persona analysis and customer interviews
on new brand

• Route traffic p new, 1-p old (contro...
A/B Testing Report and significance

p +/- (1.65 x SE)

https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html
...
Network Diffusion

Atlassian JIRA to OS communities.
SEO / SEM limitations — budget and effectiveness.
Wednesday, 6 Novemb...
Power law

Think 80/20 rule.
Store revenue, traffic, disk usage, number products, etc.
http://www.hpl.hp.com/research/idl/p...
Continuous Delivery
• Feature flag everything and gently deploy
• Deploy daily — huge difference!
• Read 12factor.net (dev,...
Ask permission, get feedback

Wednesday, 6 November 13
User Stories

Wednesday, 6 November 13
Commutative flow

Wednesday, 6 November 13
Startup Friendly Stack
•

Use Ruby Sinatra (or Clojure) +
CoffeeScript + Backbone +
Handlebars on Heroku

•

Nesta for CMS...
Peopleware
•

Do learn to code simple website
changes and growth hacking

•

Learn some stats

•

Do find a technical co-fo...
Upcoming SlideShare
Loading in...5
×

Building Products Quantitatively

998

Published on

Presentation for the Founder Institute mentoring session.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
998
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Building Products Quantitatively

    1. 1. Building Products Quantitatively in SaaS Businesses Soren Harner, Bigcommerce © 2003 – 2013 Bigcommerce Pty. Ltd. Wednesday, 6 November 13
    2. 2. About me 2012 VP Engineering Stage 2 Startup Venture-backed $35MM 2010 Founder & Datavore Stage 1 Startup Bookstrapped 2007 VP Engineering Stage 2 Startup 2000 Search & Analytics, Stage 2 Startup Stage 3 Acquisition Venture-backed > $80MM Director Engineering Wednesday, 6 November 13
    3. 3. Hill Climbing Wednesday, 6 November 13
    4. 4. SaaS Business Levers • • • • • • Cost to acquire customers (CAC) Average Revenue per User (ARPU) Churn • Goal: Negative revenue retention (upgrades) Conversion (CVN) by channel • Payback Life time value (LTV) • • LTV = ARPU/Churn LTV / CAC > 1 is success! Monthly Recurring Revenue (MRR) • Magic Number Wednesday, 6 November 13 http://blog.asmartbear.com/coc-cancellation-saas-metric.html
    5. 5. LTV / CAC Model “successful” 1.2 0.9 0.6 0.3 0 Q1-10 Q2-10 Q3-10 Q4-10 Q1-11 Q2-11 Q3-11 Q4-11 To IPO range is maybe 4 to 6 LTV / CAC today Fine print: Like all metrics use to track, not as strategy Wednesday, 6 November 13
    6. 6. Tested hundreds of versions Wednesday, 6 November 13
    7. 7. Signup, upgrade, onboarding • Invest in a great experience • Test with usertesting.com • Track the funnel: • Number of trials • Leads to Trials to Paid by weekly cohort • Onboarding success Wednesday, 6 November 13
    8. 8. Product UX Testing Numbers don’t tell you how to improve. Workshops (personas) Concepting (sketches) Prototypes Qualitative (Axure) You can use an iPad at a Cafe and show keynote. Key point is to test mockups on people first. Wednesday, 6 November 13 Quantitative (Loop11)
    9. 9. Feedback Wednesday, 6 November 13
    10. 10. Measure Product Success • Customer success • • • • • Wednesday, 6 November 13 Time to first sale Time to launch Trend monthly or weekly cohorts Engagement score during trial Churn and conversion predictive models
    11. 11. Feature KPI • Before you build a feature, define a success KPI • • Wednesday, 6 November 13 e.g. Number of product reviews completed e.g. 30% fewer abandoned carts • e.g. completing task more quickly • Add instrumentation before and after • Deploy to subset of customers to test • Consider Mixpanel
    12. 12. Split Test Important Things • Persona analysis and customer interviews on new brand • Route traffic p new, 1-p old (control) • Measure effect on conversion with 95% confidence interval of a difference and ZScore Wednesday, 6 November 13
    13. 13. A/B Testing Report and significance p +/- (1.65 x SE) https://developer.amazon.com/sdk/ab-testing/reference/ab-math.html Wednesday, 6 November 13
    14. 14. Network Diffusion Atlassian JIRA to OS communities. SEO / SEM limitations — budget and effectiveness. Wednesday, 6 November 13
    15. 15. Power law Think 80/20 rule. Store revenue, traffic, disk usage, number products, etc. http://www.hpl.hp.com/research/idl/papers/ranking/ranking.html Wednesday, 6 November 13
    16. 16. Continuous Delivery • Feature flag everything and gently deploy • Deploy daily — huge difference! • Read 12factor.net (dev, staging, production) • Monitor (i.e. NewRelic) • Practice BDD & continuous integration • Always have a recovery plan Wednesday, 6 November 13
    17. 17. Ask permission, get feedback Wednesday, 6 November 13
    18. 18. User Stories Wednesday, 6 November 13
    19. 19. Commutative flow Wednesday, 6 November 13
    20. 20. Startup Friendly Stack • Use Ruby Sinatra (or Clojure) + CoffeeScript + Backbone + Handlebars on Heroku • Nesta for CMS and learn textmate, Haml & Sass, see https://github.com/ sharner/Workshop-Site • HTML5 mobile first, responsive design • Use Mailchimp, Wufoo, SurveyMonkey • R for data (http://r-project.org) — treat as code, put in Github • Log events to understand customer success (see Heroku Plugins) Wednesday, 6 November 13
    21. 21. Peopleware • Do learn to code simple website changes and growth hacking • Learn some stats • Do find a technical co-founder • Don’t offshore • Don’t churn people • Do find designer who codes frontend and does user testing Wednesday, 6 November 13
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×