Startup analytics

517 views

Published on

Presentation given to the Gener8tor portfolio companies on 1/31/2013.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
517
On SlideShare
0
From Embeds
0
Number of Embeds
55
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Startup analytics

  1. 1. STARTUP ANALYTICSGetting Started Down the Path to Understanding Your Business and Your Users Dale Beermann Chief Technology and Analytics Officer dale@studyblue.com
  2. 2. THE GOAL OF ANALYTICS: IMPROVING YOUR BUSINESS BYANSWERING AND ACTING ON QUESTIONSWith every question answered, ask yourself if it is the desired result. If not, determine what needs to be done to improve it.
  3. 3. BUSINESS METRICS VERSUS USAGE METRICSYou should always be reporting on your business metrics.Analytics is the way to understand what is driving them. Effectively, business metrics are the aggregate result of your usage metrics.
  4. 4. BUSINESS METRICSThe ultimate goal of business metrics is to evaluate the healthof your business. Examples: How fast is your business growing? What is your churn rate? What is your cost per acquisition for each channel? What is your Average Revenue per Active User?
  5. 5. USAGE METRICSThe ultimate goal of usage metrics is to evaluate the health ofyour product. Examples: What percentage of your users are realizing your value propositions? Is your new feature reaching the expected audience? What percentage of users make it through the onboarding process? What percentage of users are using social channels?
  6. 6. INFLUENCING BUSINESS METRICSKnow the answers to your high-level business metrics beforedigging into your usage.Use your usage metrics to determine how you can influenceyour business metrics.
  7. 7. BEST PRACTICES
  8. 8. THE RIGHT TIME TO STARTHave you found your product/market fit? There may be some high level business metrics that help you get there, but don’t start your analysis on a product that is going through a massive amount of change.
  9. 9. WHERE TO STARTHave you filled out a Business Model Canvas? What are yourbusiness’ most important metrics? How well are each of your customer segments doing when it comes to realizing your value propositions? Take your value propositions and work backwards through the paths that your users take to get there.
  10. 10. DEVELOP GOOD HABITSMake Analytics a core part of your development workflow.Ensure you are creating both good behavioral habits as well asgood programming habits.
  11. 11. GOOD BEHAVIORAL HABITSReview your metrics on a regular basisContinually log changes that are going into your product There will inevitably be a point in the future where you ask yourself what happened six months ago to influence a particular metric.
  12. 12. GOOD PROGRAMMING HABITSCreate guidelines and tools that require you to implementmetrics as you build out your software E.g. Use abstract click handlers that can be easily refactored: display.addClassesHandler(new SBClickHandler(SBAnalytic.HOME_FIND_CLICK) { @Override public void doOnClick(ClickEvent event) { ... } });
  13. 13. AVOID VANITY METRICSPage views dont matter (impressions may).Time On Site can be interesting, but doesnt necessarilyconvey usage.It’s very difficult to influence metrics like Page Views or Timeon Site. Attempting to do so will be a waste of your time.
  14. 14. FOCUS ON ACTIONABLE METRICSThese are going to be different for every business.Again, you want to find the metrics that mean the most toyour company and determine how you can influence them.
  15. 15. MAXIMIZING A METRIC CAN HAVE SIDE EFFECTSProviding multiple options splits your usage between them.Similarly, forcing users down one particular path means theycan’t take another. This can arise in subtle ways.In some cases, such as with a payment page, you may be ableto find the optimal solution without many side affects.Ask yourself: What user segments are affected bythis change? Will any side effects be worth it?
  16. 16. IMPLEMENTATION
  17. 17. CAVEAT: I DO NOT SUBSCRIBE TO THE IDEATHAT YOU SHOULD LIMIT WHAT YOU TRACK. If you are smart about how you’re doing your analysis, you will not fall into the trap of “analysis paralysis.”
  18. 18. START WITH GOOGLE ANALYTICSIt’s free and you can throw everything at it without worryingabout usage tiers.We don’t use the high level (vanity) metrics for much. Rather,by sending our events through Google Analytics, we have theability to answer a lot of questions.
  19. 19. GETTING THE MOST OUT OF GOOGLE ANALYTICSTrack all of your events (views, clicks, actions). This isn’t limited to your click stream. Track final events for workflows (e.g. completed_onboarding). This allows you to create Advanced Segments for those events.Set up profiles for each platform (web, iOS, Android, etc.). You’re going to have very different usage patterns for each platform, and they should be analyzed separately.
  20. 20. GETTING THE MOST OUT OF GOOGLE ANALYTICSMake use of custom variables. At the very least, you should be setting your (non personally-identifiable) user ID as one of the variables. This will let you find some per-user data that is otherwise difficult with Google Analytics. If you have organizational data, or if your users are segmented in pre-defined ways, this can help look at those segments more closely.
  21. 21. I’M TRACKING MY EVENTS. NOW WHAT?Funnel Analysis The goal of a funnel analysis is to determine where your users are falling off. Take one of your core metrics and walk through the steps it takes to get there.
  22. 22. FUNNEL ANALYSIS EXAMPLEStudyBlue and IndexableContent We want to maximize the amount of content created that is “paired” with a class. How does that happen?
  23. 23. HOW DO YOU IMPROVE YOUR FUNNELS?Think about how can you change an experience to improvethe end result. Sometimes this is as simple as changing a button’s color or using a modal popup (while thinking about the side effects).A/B Testing A/B Testing can be a reliable way to evaluate multiple paths. Caveat: Do your homework and understand statistical significance. Learn what a chi-squared test is.
  24. 24. TOOLS FOR FUNNEL ANALYSISGoogle Analytics does make it possible to do some of this. Their goal conversions are annoying if you don’t use page views the way they expect. Create advanced segments for users with particular events.Other good for-pay tools are KissMetrics and Mixpanel.Roll your own. In all honesty, doing this stuff yourself isn’t that hard.
  25. 25. TANGENT: YOUR OWN IMPLEMENTATIONYou’ll want to use partitioned tables under the hood (if yourdata store supports it). In postgresql, we use triggers to write data to the correct table. Queries then only hit the necessary tables for the time span you’ve defined.We got away with a table per week for about 5 years. Ourtable schema: user_id, session_id, platform, activity_id, activity_timestamp, activity_detail
  26. 26. ADVANCED ANALYTICS
  27. 27. COHORT ANALYSISA cohort is a set of users grouped in a particular fashion.Typical cohorts are time-based (week of registration). Cohortscan also be based on acquisition campaigns (e.g. Adwords vs.Direct vs. SEO).The purpose of a cohort analysis is to understand userretention and if your changes are making an impact betweencohorts.
  28. 28. WHY COHORT ANALYSISMost educated investors are going to ask for cohort analyses.Cohort analyses, and their corresponding retention rates helpdetermine: Engagement levels. Are you a one-and-done sort of site? Churn rates. If users aren’t coming back to your site, or if churn is higher than acquisition, your site will not grow. Quantifying the value of your existing userbase.
  29. 29. A COHORT ANALYSIS EXAMPLESadly, I can’t provide some of our own data here. But this iswhat your cohort analyses will look like:
  30. 30. COHORT ANALYSIS QUERIESIn Posgtresql: crosstab.In MySQL: Pivot Tables (still pretty manual).In everything else: pull your data into one of the above. Orwrite a lot of code.
  31. 31. THE HOLY GRAILA full fledged Customer Relationship Management systemdriven from your analytics solutions: Adaptive in-app user education Drip email campaigns Churn prediction Re-engagement
  32. 32. QUESTIONS?

×