Make Better Decisions!Using Metrics to WinHiten ShahPlusConf • December 7th, 2010
Ferrari F430 Spider
152 miles per hour
Most Startups Crash and Burn
Let’s talk about what else is easy.
Stop measuring what’s easy.
Start measuring what matters.
Original Hypothesis for KISSinsights“ Our hypothesis is that product manager type people have a problem doing fast/effective/frequent customer research.”
Are the metrics you aretracking helping you make decisions?
Example SaaS Web App• People sign up and pay online• They ﬁrst sign up for a 7 day free trial• Then they pay $97 per month
Example SaaS Web App • People sign up and pay online • They ﬁrst sign up for a 7 day free trial • Then they pay $97 per monthWhat are your assumptions?
Example SaaS Web App Macro StepsVisit Site Sign Up Activate Upgrade
Software as a Service Product (Assumptions)Key Actions % of people # of people Visit Site 100% 1000 Sign Up 10% 100 Activate 5% 50 Upgrade 1% 10 Revenue $970 per month
Baseline these key actions and assumptions.
Software as a Service Product (Actual Baseline)Key Actions % of people # of people Visit Site 100% 1000 Sign Up 3.6% 36 Activate 1.2% 12 Upgrade 0.4% 4 Revenue $388 per month
What would you try to improve?Key Actions % of people # of people Visit Site 100% 1000 Sign Up 3.6% 36 Activate 1.2% 12 Upgrade 0.4% 4 Revenue $388 per month
Software as a Service Product (Video Test)Key Actions % of people # of people Visit Site 100% 1000 Sign Up 6.5% 65 Activate 1.8% 18 Upgrade 0.3% 3 Revenue $291 per month vs. $388 baseline :(
Get inspired by what other’s have done
Software as a Service Product (Inspired Test)Key Actions % of people # of people Visit Site 100% 1000 Sign Up 7.7% 77 Activate 2.8% 28 Upgrade 1.2% 12 Revenue $1164 per month vs. $388 baseline :)
Software as a Service Product (Compare all 3)Key Actions Baseline Video Test Inspired Test Visit Site 100% 100% 100% Sign Up 3.6% 6.5% 7.7% Activate 1.2% 1.8% 2.8% Upgrade 0.4% 0.3% 1.2% Revenue $388 per $291 per $1164 per month month month
The Startup Pyramid The Race to the Top Growth Transition to Growth Product / Market Fit Sean Ellis: Startup-Marketing.com
AARRRCreated by Dave McClure - 500startups.com Designed by 1. ACQUISITION Customers come from various channels 2. ACTIVATION Customers use product for first time YOUR PRODUCT 3. RETENTION 4. REFERRAL Customers come back multiple times Customers refer others 5. REVENUE Customers make your business money
Minimize TOTAL time through the loop IDEASLearn Faster Build Faster LEARN BUILD Unit TestsSplit TestsCustomer Interviews Usability TestsCustomer Development Continuous IntegrationFive Whys Root Cause Incremental DeploymentAnalysis Free & Open-SourceCustomer Advisory Board ComponentsFalsifiable Hypotheses DATA CODE Cloud ComputingProduct Owner Cluster Immune System Just-in-time ScalabilityAccountability MEASURE RefactoringCustomer Archetypes Measure FasterCross-functional Teams Developer SandboxSemi-autonomous Teams Split Tests Funnel Analysis Minimum Viable ProductSmoke Tests Clear Product Owner Cohort Analysis Continuous Deployment Net Promoter Score Usability Tests Search Engine Marketing Real-time Monitoring Real-Time Alerting Customer Liaison Predictive Monitoring Eric Ries: StartupLessonsLearned.com
Lean Startup Data-Driven Process
No Departments, Just Two Teams ? Problem Team What are the ☼ Solution Team Find the most important minimum solution problems to solve? for a given problem.