SlideShare a Scribd company logo
Testing, measuring and
        metrics
     Soren Harner, Bigcommerce
About me
2012 VP Engineering                   Stage 2 Startup
                                      Venture-backed
                                      $35MM


                                      Stage 1 Startup
2010 Founder & Datavore
                                      Bookstrapped




2007 VP Engineering                   Stage 2 Startup




                                      Stage 2 Startup
2000 Search & Analytics,              Stage 3 Acquisition
 Director Engineering                 Venture-backed > $80MM
Business Metrics
•   Cost to acquire customers (CAC)
•   Average Revenue per User (ARPU)
•   Churn
    •   Avoidable & unavoidable
•   Conversion (CVN)
    •   Compare Search, Display, Radio,
        etc.
•   Life time value (LTV)
    •   LTV = ARPU/Churn
    •   LTV / CAC > 1 is success!
•   Monthly Recurring Revenue (MRR)
LTV / CAC
1.2
                                                                              Model
                                                                              “successful”
0.9


0.6


0.3


 0
 Q1-10   Q2-10      Q3-10     Q4-10      Q1-11     Q2-11      Q3-11     Q4-11



      To IPO range is maybe 4 to 6 LTV / CAC today
                 Fine print: Like all metrics use to track, not as strategy
Split Testing Website
• Persona analysis and customer interviews
  on new brand
• Of new traffic, 50% old, 50% new
• Measure effect on conversion: looked at
  new trials after 50/50 coming from old and
  new to determine lift with 95% confidence
  interval of 20-30%
Product UX Testing


Workshops         Concepting                     Qualitative   Quantitative
                                 Prototypes
(personas)        (sketches)                      (Axure)       (Loop11)




             You can use an iPad at a Cafe and show keynote.
               Key point is to test mockups on people first.
Continuous Delivery

• Feature flag everything and gently deploy
• Weekly deploys (should be daily)
• Read 12factor.net (dev, staging, production)
• Monitor (i.e. NewRelic)
• Practice BDD & continuous integration
Ask permission, get feedback
Product Metrics
•   Customer success                   TTL        TTFS


    •   Time to first sale   Aug


    •   Time to launch      July


•   Trend monthly or        June
    weekly cohorts
                            May
•   Engagement score
    during trial                   0    25   50    75    100
Feature KPI
•   Before you build a             •   e.g. completing task
    feature, define a success           more quickly
    KPI
                               •   Add instrumentation
    •   e.g. Number of             before and after
        product reviews
        completed              •   Deploy to subset of
                                   customers to test
    •   e.g. 30% fewer
        abandoned carts        •   Consider Mixpanel
Startup Friendly Stack
•   Use Ruby Sinatra (or Clojure) +
    CoffeeScript + Backbone + Handlebars
    on Heroku


•   Nesta for CMS and learn textmate,
    Haml & Sass, see https://github.com/
    sharner/Workshop-Site


•   HTML5 mobile first, responsive design


•   Use Mailchimp & google forms


•   Log events to understand customer
    success (see Heroku Plugins)
Peopleware
•   Technical co-founder
    challenges

•   Don’t offshore

•   Don’t churn people

•   Find designer who codes
    frontend and does user
    testing

More Related Content

Similar to SaaS and Product Metrics

Building Products Quantitatively
Building Products QuantitativelyBuilding Products Quantitatively
Building Products QuantitativelySoren Harner
 
Unleashing change impact mining for sap dev ops
Unleashing change impact mining for sap dev opsUnleashing change impact mining for sap dev ops
Unleashing change impact mining for sap dev opsRahul Jacob
 
What is DevOps? How can it impact my Customers and my Business
What is DevOps? How can it impact my Customers and my BusinessWhat is DevOps? How can it impact my Customers and my Business
What is DevOps? How can it impact my Customers and my BusinessQualitest
 
Dashlane Mission Teams
Dashlane Mission TeamsDashlane Mission Teams
Dashlane Mission TeamsDashlane
 
Sledgehammer to Fine Brush for QA
Sledgehammer to Fine Brush for QASledgehammer to Fine Brush for QA
Sledgehammer to Fine Brush for QAShelley Lambert
 
Agile product development and project management with Kanban
Agile product development and project management with KanbanAgile product development and project management with Kanban
Agile product development and project management with KanbanAlberto Caeiro, CSPO, CSM, PMP
 
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)Samuel Chin, PMP, CSM
 
Process Guidelines V2
Process Guidelines V2Process Guidelines V2
Process Guidelines V2Imaginea
 
Managing an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product LeaderManaging an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product LeaderProduct School
 
Agile metrics at-pmi bangalore
Agile metrics at-pmi bangaloreAgile metrics at-pmi bangalore
Agile metrics at-pmi bangaloreBimlesh Gundurao
 
Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.Worksoft
 
Simplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless ToolsSimplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless ToolsSauce Labs
 
Sanket Kumar Lade Resume
Sanket Kumar Lade ResumeSanket Kumar Lade Resume
Sanket Kumar Lade ResumeSanket Lade
 
Quantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROIQuantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROIDevOps for Enterprise Systems
 
Sales Wave Apps - Partner Training
Sales Wave Apps - Partner TrainingSales Wave Apps - Partner Training
Sales Wave Apps - Partner TrainingSalesforce Partners
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardTechWell
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of ExperimentationOptimizely
 
Evolving the Product Management Process to Match Company Growth
Evolving the Product Management Process to Match Company GrowthEvolving the Product Management Process to Match Company Growth
Evolving the Product Management Process to Match Company GrowthSVPMA
 
ROI at the bug factory - Goldratt & throughput (2004)
ROI at the bug factory - Goldratt & throughput (2004)ROI at the bug factory - Goldratt & throughput (2004)
ROI at the bug factory - Goldratt & throughput (2004)Neil Thompson
 

Similar to SaaS and Product Metrics (20)

Building Products Quantitatively
Building Products QuantitativelyBuilding Products Quantitatively
Building Products Quantitatively
 
Unleashing change impact mining for sap dev ops
Unleashing change impact mining for sap dev opsUnleashing change impact mining for sap dev ops
Unleashing change impact mining for sap dev ops
 
SAFe and DevOps - better together
SAFe and DevOps - better togetherSAFe and DevOps - better together
SAFe and DevOps - better together
 
What is DevOps? How can it impact my Customers and my Business
What is DevOps? How can it impact my Customers and my BusinessWhat is DevOps? How can it impact my Customers and my Business
What is DevOps? How can it impact my Customers and my Business
 
Dashlane Mission Teams
Dashlane Mission TeamsDashlane Mission Teams
Dashlane Mission Teams
 
Sledgehammer to Fine Brush for QA
Sledgehammer to Fine Brush for QASledgehammer to Fine Brush for QA
Sledgehammer to Fine Brush for QA
 
Agile product development and project management with Kanban
Agile product development and project management with KanbanAgile product development and project management with Kanban
Agile product development and project management with Kanban
 
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)
Products and Value: An Agile Perspective BY Matt Nudelmann (GUEST PRESENTER)
 
Process Guidelines V2
Process Guidelines V2Process Guidelines V2
Process Guidelines V2
 
Managing an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product LeaderManaging an Experimentation Platform by LinkedIn Product Leader
Managing an Experimentation Platform by LinkedIn Product Leader
 
Agile metrics at-pmi bangalore
Agile metrics at-pmi bangaloreAgile metrics at-pmi bangalore
Agile metrics at-pmi bangalore
 
Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.Shift Left Quality Assurance: How to do it. Why it matters.
Shift Left Quality Assurance: How to do it. Why it matters.
 
Simplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless ToolsSimplify Salesforce Testing with AI-Driven Codeless Tools
Simplify Salesforce Testing with AI-Driven Codeless Tools
 
Sanket Kumar Lade Resume
Sanket Kumar Lade ResumeSanket Kumar Lade Resume
Sanket Kumar Lade Resume
 
Quantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROIQuantifying DevOps Adoption Empirically for Demonstrable ROI
Quantifying DevOps Adoption Empirically for Demonstrable ROI
 
Sales Wave Apps - Partner Training
Sales Wave Apps - Partner TrainingSales Wave Apps - Partner Training
Sales Wave Apps - Partner Training
 
Continuous Performance Testing: The New Standard
Continuous Performance Testing: The New StandardContinuous Performance Testing: The New Standard
Continuous Performance Testing: The New Standard
 
[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation[Webinar] Visa's Journey to a Culture of Experimentation
[Webinar] Visa's Journey to a Culture of Experimentation
 
Evolving the Product Management Process to Match Company Growth
Evolving the Product Management Process to Match Company GrowthEvolving the Product Management Process to Match Company Growth
Evolving the Product Management Process to Match Company Growth
 
ROI at the bug factory - Goldratt & throughput (2004)
ROI at the bug factory - Goldratt & throughput (2004)ROI at the bug factory - Goldratt & throughput (2004)
ROI at the bug factory - Goldratt & throughput (2004)
 

More from Soren Harner

Intelligent Application Networks with Mule and Tensorflow
Intelligent Application Networks with Mule and TensorflowIntelligent Application Networks with Mule and Tensorflow
Intelligent Application Networks with Mule and TensorflowSoren Harner
 
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...Soren Harner
 
How Developers Save the Planet
How Developers Save the PlanetHow Developers Save the Planet
How Developers Save the PlanetSoren Harner
 
Cundal gathering june 2011
Cundal gathering june 2011Cundal gathering june 2011
Cundal gathering june 2011Soren Harner
 
Software on the High Seas
Software on the High SeasSoftware on the High Seas
Software on the High SeasSoren Harner
 
Scaling atlassian os v4
Scaling atlassian os v4Scaling atlassian os v4
Scaling atlassian os v4Soren Harner
 
Sustainable purchasing
Sustainable purchasingSustainable purchasing
Sustainable purchasingSoren Harner
 

More from Soren Harner (10)

Intelligent Application Networks with Mule and Tensorflow
Intelligent Application Networks with Mule and TensorflowIntelligent Application Networks with Mule and Tensorflow
Intelligent Application Networks with Mule and Tensorflow
 
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...
How Augmented Reality Will Improve Our Interactions with Computers from SVVR ...
 
How Developers Save the Planet
How Developers Save the PlanetHow Developers Save the Planet
How Developers Save the Planet
 
Retrospectives
RetrospectivesRetrospectives
Retrospectives
 
Cundal gathering june 2011
Cundal gathering june 2011Cundal gathering june 2011
Cundal gathering june 2011
 
Software on the High Seas
Software on the High SeasSoftware on the High Seas
Software on the High Seas
 
Scaling atlassian os v4
Scaling atlassian os v4Scaling atlassian os v4
Scaling atlassian os v4
 
CoffeeScript
CoffeeScriptCoffeeScript
CoffeeScript
 
Sustainable purchasing
Sustainable purchasingSustainable purchasing
Sustainable purchasing
 
KNOW CHANGE
KNOW CHANGEKNOW CHANGE
KNOW CHANGE
 

Recently uploaded

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxDavid Michel
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastUXDXConf
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaCzechDreamin
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIES VE
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKUXDXConf
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...FIDO Alliance
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...CzechDreamin
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfChristopherTHyatt
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxJennifer Lim
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfFIDO Alliance
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeCzechDreamin
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...CzechDreamin
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1DianaGray10
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2DianaGray10
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsStefano
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyUXDXConf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024Stephanie Beckett
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekCzechDreamin
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomCzechDreamin
 

Recently uploaded (20)

Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptxUnpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
Unpacking Value Delivery - Agile Oxford Meetup - May 2024.pptx
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
IESVE for Early Stage Design and Planning
IESVE for Early Stage Design and PlanningIESVE for Early Stage Design and Planning
IESVE for Early Stage Design and Planning
 
Connecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAKConnecting the Dots in Product Design at KAYAK
Connecting the Dots in Product Design at KAYAK
 
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
Secure Zero Touch enabled Edge compute with Dell NativeEdge via FDO _ Brad at...
 
Syngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdfSyngulon - Selection technology May 2024.pdf
Syngulon - Selection technology May 2024.pdf
 
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
SOQL 201 for Admins & Developers: Slice & Dice Your Org’s Data With Aggregate...
 
Agentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdfAgentic RAG What it is its types applications and implementation.pdf
Agentic RAG What it is its types applications and implementation.pdf
 
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptxWSO2CONMay2024OpenSourceConferenceDebrief.pptx
WSO2CONMay2024OpenSourceConferenceDebrief.pptx
 
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdfLinux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
Linux Foundation Edge _ Overview of FDO Software Components _ Randy at Intel.pdf
 
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi IbrahimzadeFree and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
Free and Effective: Making Flows Publicly Accessible, Yumi Ibrahimzade
 
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
Behind the Scenes From the Manager's Chair: Decoding the Secrets of Successfu...
 
UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1UiPath Test Automation using UiPath Test Suite series, part 1
UiPath Test Automation using UiPath Test Suite series, part 1
 
UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2UiPath Test Automation using UiPath Test Suite series, part 2
UiPath Test Automation using UiPath Test Suite series, part 2
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
AI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří KarpíšekAI revolution and Salesforce, Jiří Karpíšek
AI revolution and Salesforce, Jiří Karpíšek
 
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone KomSalesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
Salesforce Adoption – Metrics, Methods, and Motivation, Antone Kom
 

SaaS and Product Metrics

  • 1. Testing, measuring and metrics Soren Harner, Bigcommerce
  • 2. About me 2012 VP Engineering Stage 2 Startup Venture-backed $35MM Stage 1 Startup 2010 Founder & Datavore Bookstrapped 2007 VP Engineering Stage 2 Startup Stage 2 Startup 2000 Search & Analytics, Stage 3 Acquisition Director Engineering Venture-backed > $80MM
  • 3. Business Metrics • Cost to acquire customers (CAC) • Average Revenue per User (ARPU) • Churn • Avoidable & unavoidable • Conversion (CVN) • Compare Search, Display, Radio, etc. • Life time value (LTV) • LTV = ARPU/Churn • LTV / CAC > 1 is success! • Monthly Recurring Revenue (MRR)
  • 4. LTV / CAC 1.2 Model “successful” 0.9 0.6 0.3 0 Q1-10 Q2-10 Q3-10 Q4-10 Q1-11 Q2-11 Q3-11 Q4-11 To IPO range is maybe 4 to 6 LTV / CAC today Fine print: Like all metrics use to track, not as strategy
  • 5.
  • 6. Split Testing Website • Persona analysis and customer interviews on new brand • Of new traffic, 50% old, 50% new • Measure effect on conversion: looked at new trials after 50/50 coming from old and new to determine lift with 95% confidence interval of 20-30%
  • 7. Product UX Testing Workshops Concepting Qualitative Quantitative Prototypes (personas) (sketches) (Axure) (Loop11) You can use an iPad at a Cafe and show keynote. Key point is to test mockups on people first.
  • 8. Continuous Delivery • Feature flag everything and gently deploy • Weekly deploys (should be daily) • Read 12factor.net (dev, staging, production) • Monitor (i.e. NewRelic) • Practice BDD & continuous integration
  • 10. Product Metrics • Customer success TTL TTFS • Time to first sale Aug • Time to launch July • Trend monthly or June weekly cohorts May • Engagement score during trial 0 25 50 75 100
  • 11. Feature KPI • Before you build a • e.g. completing task feature, define a success more quickly KPI • Add instrumentation • e.g. Number of before and after product reviews completed • Deploy to subset of customers to test • e.g. 30% fewer abandoned carts • Consider Mixpanel
  • 12. Startup Friendly Stack • Use Ruby Sinatra (or Clojure) + CoffeeScript + Backbone + Handlebars on Heroku • Nesta for CMS and learn textmate, Haml & Sass, see https://github.com/ sharner/Workshop-Site • HTML5 mobile first, responsive design • Use Mailchimp & google forms • Log events to understand customer success (see Heroku Plugins)
  • 13. Peopleware • Technical co-founder challenges • Don’t offshore • Don’t churn people • Find designer who codes frontend and does user testing

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n