Download as KEY, PPTX
Uploaded byHiten Shah
Make Better Decisions!
The document provides an overview of using metrics to make better decisions for startups. It discusses moving from tracking vanity metrics that are easy to measure to actionable metrics that matter. An example is given of a SaaS web app that tracks key metrics like sign-ups, activations and upgrades at each step. The actual baseline results are compared to assumptions, and different tests are discussed, like a video test that decreased conversions and an inspired test that increased revenue. The importance of learning faster through tests and data is emphasized, as is measuring faster and minimizing the time to get customers through the process.
More Related Content
Make Better Decisions!
- 1. Make Better Decisions! UsingMetrics to Win Hiten Shah PlusConf • December 7th, 2010
- 2.
- 3. 152 miles perhour
- 4. Most Startups Crashand Burn
- 5. Let’s talk aboutwhat else is easy.
- 8.
- 9. Start measuring whatmatters.
- 10.
- 11.
- 12. Original Hypothesis forKISSinsights “ Our hypothesis is that product manager type people have a problem doing fast/effective/frequent customer research.”
- 13. Are the metricsyou are tracking helping you make decisions?
- 14. Example SaaS WebApp • People sign up and pay online • They first sign up for a 7 day free trial • Then they pay $97 per month
- 15. Example SaaS WebApp • People sign up and pay online • They first sign up for a 7 day free trial • Then they pay $97 per month What are your assumptions?
- 16. Example SaaS WebApp Macro Steps Visit Site Sign Up Activate Upgrade
- 17. Software as aService Product (Assumptions) Key Actions % of people # of people Visit Site 100% 1000 Sign Up 10% 100 Activate 5% 50 Upgrade 1% 10 Revenue $970 per month
- 18. Baseline these keyactions and assumptions.
- 19. Software as aService Product (Actual Baseline) Key Actions % of people # of people Visit Site 100% 1000 Sign Up 3.6% 36 Activate 1.2% 12 Upgrade 0.4% 4 Revenue $388 per month
- 20. What would youtry to improve? Key Actions % of people # of people Visit Site 100% 1000 Sign Up 3.6% 36 Activate 1.2% 12 Upgrade 0.4% 4 Revenue $388 per month
- 21. Software as aService Product (Video Test) Key Actions % of people # of people Visit Site 100% 1000 Sign Up 6.5% 65 Activate 1.8% 18 Upgrade 0.3% 3 Revenue $291 per month vs. $388 baseline :(
- 22. Get inspired bywhat other’s have done
- 23. Software as aService Product (Inspired Test) Key Actions % of people # of people Visit Site 100% 1000 Sign Up 7.7% 77 Activate 2.8% 28 Upgrade 1.2% 12 Revenue $1164 per month vs. $388 baseline :)
- 24. Software as aService Product (Compare all 3) Key Actions Baseline Video Test Inspired Test Visit Site 100% 100% 100% Sign Up 3.6% 6.5% 7.7% Activate 1.2% 1.8% 2.8% Upgrade 0.4% 0.3% 1.2% Revenue $388 per $291 per $1164 per month month month
- 25. The Startup Pyramid The Race to the Top Growth Transition to Growth Product / Market Fit Sean Ellis: Startup-Marketing.com
- 26. AARRR Created by DaveMcClure - 500startups.com Designed by 1. ACQUISITION Customers come from various channels 2. ACTIVATION Customers use product for first time YOUR PRODUCT 3. RETENTION 4. REFERRAL Customers come back multiple times Customers refer others 5. REVENUE Customers make your business money
- 27. Minimize TOTAL timethrough the loop IDEAS Learn Faster Build Faster LEARN BUILD Unit Tests Split Tests Customer Interviews Usability Tests Customer Development Continuous Integration Five Whys Root Cause Incremental Deployment Analysis Free & Open-Source Customer Advisory Board Components Falsifiable Hypotheses DATA CODE Cloud Computing Product Owner Cluster Immune System Just-in-time Scalability Accountability MEASURE Refactoring Customer Archetypes Measure Faster Cross-functional Teams Developer Sandbox Semi-autonomous Teams Split Tests Funnel Analysis Minimum Viable Product Smoke Tests Clear Product Owner Cohort Analysis Continuous Deployment Net Promoter Score Usability Tests Search Engine Marketing Real-time Monitoring Real-Time Alerting Customer Liaison Predictive Monitoring Eric Ries: StartupLessonsLearned.com
- 28. Lean Startup Data-DrivenProcess
- 29. No Departments, JustTwo Teams ? Problem Team What are the ☼ Solution Team Find the most important minimum solution problems to solve? for a given problem.
- 30. 15 - 45minute team meetings (M,W,F)
- 31. “Your Focus NeedsMore Focus”
- 32. Questions? Got questions? Wantadvice? Email Me! hshah@kissmetrics.com
Editor's Notes
- #2 Introduce myself.\n
- #3 How many of your know how to drive?\nHow many of you would like to drive this car?\n
- #4 How about blind-folded at 152 miles per hour?\nWell, that’s what most of us have been doing with our startups!\nSo...\n
- #5 We crash and burn.\nYou might even crash and burn if you aren’t blind-folded!\nSame with a startup, it’s EASY to fail. We all know that.\n
- #6 What’s easy?\n
- #7 It’s easy to add Google Analytics to your site and see reports.\nI am sure, most of our Google Analytics graphs don’t look like this.\nTraditional web analytics shows you page views, visits, page per visit, time on site, etc...\nThese are all easy metrics to get.\n
- #8 It’s also easy to track click through rates on ads.\n
- #9 Measuring what’s easy will increase you likelihood of failure.\nYou’ll crash and burn and end up with a startup just like this Ferrari.\n
- #10 Measuring what matters will increase your likelihood of success.\n
- #11 Vanity metrics are easy, they make you feel good.\nBest case = Up and to the right\nEveryone trying to take credit for it.\nEither way, what do you do next?\n
- #12 Actionable metrics point out problems and lead to solutions.\nThey make your learn faster and smarter about your business.\n
- #13 “Our hypothesis is that product manager type people have a problem doing fast/effective/frequent customer research.”\nQuestion: Is this hypothesis in fact true or not?!\nTake the least amount of effort to find out...\nWe did everything from customer interviews, fake landing pages, building a minimum viable product, pivoting and now have reach product / market fit and are optimizing the business.\n
- #14 How do you know what metrics matters? \nWhen I hear a metric from someone, I always try to ask them what decision that metric helps them make.\n
- #15 People sign up and pay online\nSign up for a 7 day free trial\nPay $97 per month\n
- #16 What would you measure?\nWhat would you optimize?\nSign up? Page views? Time on site? Bounce rate? CTR on ads?\nNext step is to write these down...\n\n
- #17 \n
- #18 Create a funnel out of the user actions and add numbers.\nThese user actions and numbers are your assumptions.\n
- #19 Get real data!\n
- #20 This is the funnel after implementing tracking for each step.\nBased on the original assumptions of 1% of people upgrading...\nWe’re not there yet. So...\n\n
- #21 Try to understand your riskiest point.\nUsually if people aren’t purchasing you want to focus on that.\nIn this example, people were purchasing, but the sign up is really low.\nSo we tried...\n
- #22 Adding a video to our homepage.\nIt increased the sign up rate, which was great.\nBut it decreased revenue :(\nFocusing on all the actionable metrics enabled us to see this.\nSo...we got inspired...\n
- #23 \n
- #24 The sign up rate increased, and the revenue increased dramatically.\n\n
- #25 You want to make sure you can compare the groups of people for these tests.\n
- #26 Sean Ellis and his 3 stages of a startup\n\nWe use the Startup Pyramid to remind us where we are and what our success metrics should be.\n\nProduct / Market Fit, Transition to Growth, Growth\n
- #27 AARRR!!! Dave McClure’s pirate metrics.\n\nHelps you frame key areas of your business based on a customer’s lifecycle.\n\nWe use this for idea generation.\n
- #28 And of course, Eric’s lean startup graphic.\n\nIt’s about validated learnings about customers and increasing cycle time.\n\nSo we can build something a LOT of people want to buy.\n
- #29 The principles of build, measure, learn help us think about how to get answers.\nThe methods and process of communication and information exchange are much different in a data-driven startup.\nThe tactical stuff to build, measure and learn faster is one side of things.\nNow I’m going to talk about the business processes and what we have learned trying to apply them.\nBefore continuing...clarify business process for a startup.\n\n
- #30 Company is divided into two teams. No sales, marketing, etc...\nProblem team identifies the most important problems to solve.\nSolution team finds the minimum solution for each problem.\nProblem team is more non-technical, solution team is more technical / “makers”. There are cross-functional folks too.\n
- #31 3 team meetings every week\nMonday - Team meeting to describe most important problems and goals for the week. Metrics + Few tasks, determine if solution team has input and let deadline for ideas on Tuesday.\nWednesday - Check in on how are we doing, solutions already getting implemented? What have we learned so far?\nFriday - Postmortem on learning and peek into next week from the Problem Team.\nProblem team is 1-week ahead.\n
- #32 To do this effectively we built a tool we are internally call KISSfocus.\nA tool we built to increase autonomy and communication, focused on our validated learnings driven process.\n5 tasks per person, only 1 is active (green-lighted), not in KF then not exist, backlog.\nWe set weekly goal and metrics based on most important problems.\nWhole team has visibility across all tasks, keeps solution team focused on most important problems.\n
- #33 \n