METRICS                               LESSONS                               LEARNED                               @andreas...
Andreas Klinger      CWTFO of LOOKK       @andreasklingerSlides: http://www.slideshare.net/andreasklinger
STARTuP          LOOKK | CONDÉ NAST | 10-14-2011 | 3
STARTuPBEFORE PRODuCT  MARKET FIT                 LOOKK | CONDÉ NAST | 10-14-2011 | 4
STARTuPBEFORE PRODuCT  MARKET FIT “BLuE OCEAN”                 LOOKK | CONDÉ NAST | 10-14-2011 | 5
STARTuPBEFORE PRODuCT  MARKET FIT “BLuE OCEAN” ESS         M ADN                 LOOKK | CONDÉ NAST | 10-14-2011 | 6
www.LOOKK.com
4 Biggest (development) Problems           we had/have:                   +1                                              ...
4 Biggest (development) Problems           we had/have:                   +1                                              ...
BuZZ IS OK   YOu?NuMBERS ARE OKFEEDBACK    IS OK   MEDIA    IS OK                    LOOKK | CONDÉ NAST | 10-14-2011 | 10
BuZZ IS OK   YOu?                    SOMETHINg                    IS WRONgNuMBERS ARE OK             TRAFFIC              ...
BuZZ IS OK   YOu?                    SOMETHINg                    IS WRONgNuMBERS ARE OK             TRAFFIC              ...
vANITY METRICSAlways goes up.   ESPECIALLY:                  - Registered users                  - Facebook Fans
vANITY METRICSAlways goes up.   ESPECIALLY:                  - Registered users                  - Facebook FansNot action...
vANITY METRICS v2                             OF THIS MONTH- visitors / Month   Tell you nothing- users / Month           ...
vANITY METRICS v2                                                    OF THIS MONTH     - visitors / Month                 ...
BEFORE                              PRODuCT MARKET FIT                               FOCuS ON                             ...
BEFORE PMF FOCuS ON RETENTIONhttp://www.ashmaurya.com/2010/07/3-rules-to-             actionable-metrics/                 ...
THE REASON:                                 RETENTION                                     =                              f...
RETENTION =f(uSERHAPPINESS)Actually you don’t want to measure retention.   You want to measure user Happiness.           E...
KPI’S                               NEED TO BE                              YOuR HEALTH                                MON...
KPI’S         NEED TO BE        YOuR HEALTH          MONITOR   What numbers do actually show you the (real)           heal...
IN PRODuCT DISCOvERY                              MARKETINg                               CAN HuRT                        ...
(WRONg) MARKETINgSTuNTS (CAN) HuRT YOuR (ACTIONABLE) NuMBERS         E.g. PR BuRSTS OR COMPETITIONS         - Created visi...
DRILL DOWN                               TO REMOvE                              DATASCHMuTZ#HNLONDON | @andreasklinger
DRILL DOWN TO REMOvE    DATASCHMuTZ*    Q1: What numbers really show the health of your    Q2: How can you make them more ...
THE METRICS FAIRY SAYS…                              METRICS                               NEED                           ...
METRICS NEED TO HuRTIf the metrics you focus on do not hurt youeverytime you look at them and you are too     embarassed t...
START WITH                          AARRREXAMPLE - LOOKK                    Nice to look at. useless to work with.ACQuISIT...
“AARRR” IS THE   SOuND OF PAIN              ACQuISITION      You need a way to reach the user.     (Dave is wrong. A visit...
TOO MANY                               METRICS                                 ARE                               BAD FOR  ...
TOO MANY         METRICS           ARE         BAD FOR        YOuR EYES On your dashboards only watch those metrics you ar...
LESSONS                              LEARNED                               TOOLS#LEANCAMP | @andreasklinger
TOOLSTL;DR: All Suck.
TOOLSTL;DR: All Suck.…but are useful for certain parts:
TOOLSTL;DR: All Suck.…but are useful for certain parts:Kissmetrics, Mixpanel = People AnalysisYour own Database = drilldow...
TOOLSTL;DR: All Suck.…but are useful for certain parts:Kissmetrics, Mixpanel = People AnalysisYour own Database = drilldow...
TOOLSgoogleAnalytics                           - Referrals                           - Optimize Online Marketing,         ...
TOOLSKissMetrics                                          - Insights on Conversions                                       ...
TOOLSLOOKKButler                 - SQL = quick, Tables = cheap                 - You can drilldown to your                ...
TOOLSLOOKKButler         Team dashboards are easy to do
BuT…                                  ALWAYS REMEMBER                                     IN PRE-PMF                      ...
THANKS.                    PLEASE                    SHARE                     YOuR                   LESSONS             ...
Upcoming SlideShare
Loading in...5
×

StartUp Metrics Lesson Learned - LeanCamp Barcelona

8,052

Published on

I have given this talk at LeanCamp Barcelona and roughly the same at Hackernews London.

It's an extended and updated version of the previous LeanCamp London talk.

Feedback highly welcome.

Published in: Technology, Business
9 Comments
25 Likes
Statistics
Notes
No Downloads
Views
Total Views
8,052
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
172
Comments
9
Likes
25
Embeds 0
No embeds

No notes for slide

StartUp Metrics Lesson Learned - LeanCamp Barcelona

  1. 1. METRICS LESSONS LEARNED @andreasklinger#LEANCAMP | @andreasklinger
  2. 2. Andreas Klinger CWTFO of LOOKK @andreasklingerSlides: http://www.slideshare.net/andreasklinger
  3. 3. STARTuP LOOKK | CONDÉ NAST | 10-14-2011 | 3
  4. 4. STARTuPBEFORE PRODuCT MARKET FIT LOOKK | CONDÉ NAST | 10-14-2011 | 4
  5. 5. STARTuPBEFORE PRODuCT MARKET FIT “BLuE OCEAN” LOOKK | CONDÉ NAST | 10-14-2011 | 5
  6. 6. STARTuPBEFORE PRODuCT MARKET FIT “BLuE OCEAN” ESS M ADN LOOKK | CONDÉ NAST | 10-14-2011 | 6
  7. 7. www.LOOKK.com
  8. 8. 4 Biggest (development) Problems we had/have: +1 ? OvER NO BLOCKED uNCLEARENgINEERED DATA PROCESSES uSER vALuE AND OvER DRIvEN PROPCONCEPTED STARTuP SOLuTION? (PRE-PMF) FEATuRES CuLTuRE KANBAN BOARD SOLuTION? SOLuTION? SOLuTION? CuSTDEv FEATuRE gOINg MENTALROLLOuTS IN ON METRICS. A/B LOOKK | CONDÉ NAST | 10-14-2011 | 8
  9. 9. 4 Biggest (development) Problems we had/have: +1 ? OvER NO BLOCKED uNCLEARENgINEERED DATA PROCESSES uSER vALuE AND OvER DRIvEN PROPCONCEPTED STARTuP SOLuTION? (PRE-PMF) FEATuRES CuLTuRE KANBAN BOARD SOLuTION? SOLuTION? SOLuTION? CuSTDEv FEATuRE gOINg MENTAL +ROLLOuTS IN ON METRICS. uSERgROuP A/B IN FB LOOKK | CONDÉ NAST | 10-14-2011 | 9
  10. 10. BuZZ IS OK YOu?NuMBERS ARE OKFEEDBACK IS OK MEDIA IS OK LOOKK | CONDÉ NAST | 10-14-2011 | 10
  11. 11. BuZZ IS OK YOu? SOMETHINg IS WRONgNuMBERS ARE OK TRAFFIC SPIKESFEEDBACK IS OK NO STICKYNESS MEDIA PuSHES IS OK DON’T LIFT TRAFFIC LOOKK | CONDÉ NAST | 10-14-2011 | 11
  12. 12. BuZZ IS OK YOu? SOMETHINg IS WRONgNuMBERS ARE OK TRAFFIC SPIKESFEEDBACK IS OK NO STICKYNESS MEDIA PuSHES IS OK DON’T LIFT TRAFFIC vANITY WHAT ACTIONS WORK? LOOKK | CONDÉ NAST | 10-14-2011 | 12
  13. 13. vANITY METRICSAlways goes up. ESPECIALLY: - Registered users - Facebook Fans
  14. 14. vANITY METRICSAlways goes up. ESPECIALLY: - Registered users - Facebook FansNot actionable…
  15. 15. vANITY METRICS v2 OF THIS MONTH- visitors / Month Tell you nothing- users / Month Usually more related to external factors (PR, Search, etc)
  16. 16. vANITY METRICS v2 OF THIS MONTH - visitors / Month Tell you nothing - users / Month Usually more related to external factors (PR, Search, etc) * Tell you nothing about the health of your business or productIt’s a like a shop saying “we do good… there are now more people on the street” … and not actionable
  17. 17. BEFORE PRODuCT MARKET FIT FOCuS ON RETENTION#LEANCAMP | @andreasklinger
  18. 18. BEFORE PMF FOCuS ON RETENTIONhttp://www.ashmaurya.com/2010/07/3-rules-to- actionable-metrics/ Summary: In Discovery/validation Focus on Retention & Activation In validation/growth Focus on Revenue and Referral
  19. 19. THE REASON: RETENTION = f(uSERHAPPINESS)#LEANCAMP | @andreasklinger
  20. 20. RETENTION =f(uSERHAPPINESS)Actually you don’t want to measure retention. You want to measure user Happiness. Especially before PMF. Rentention is the best signal for userhappiness. But if you can - narrow down your KPIs to show user happiness and the health of your product. If you are multi sided (eg. buyers/sellers) segment each group seperately IF YOu WILL DO ONE THINg ONLY: FOCuS ON uSER HAPPINESS
  21. 21. KPI’S NEED TO BE YOuR HEALTH MONITOR#LEANCAMP | @andreasklinger
  22. 22. KPI’S NEED TO BE YOuR HEALTH MONITOR What numbers do actually show you the (real) health of your product? What numbers could show user happiness? E.g. Crashpadder.com (privat room rentals)calculates an activity/happiness score for each Host.And then creates groups (cohorts) for each city and compares over time.
  23. 23. IN PRODuCT DISCOvERY MARKETINg CAN HuRT YOuR NuMBERS#HNLONDON | @andreasklinger
  24. 24. (WRONg) MARKETINgSTuNTS (CAN) HuRT YOuR (ACTIONABLE) NuMBERS E.g. PR BuRSTS OR COMPETITIONS - Created visits in google Analytics - Created userentries in Database - Disappeared. - Lower your retention and activation (%) rates - Defocus you from understanding our product. Example: LOOKK We asked users to vote designers in a competition. ass about LOOKK. They just created “Dataschmutz”. useless data that obfuscates your real user behaviour
  25. 25. DRILL DOWN TO REMOvE DATASCHMuTZ#HNLONDON | @andreasklinger
  26. 26. DRILL DOWN TO REMOvE DATASCHMuTZ* Q1: What numbers really show the health of your Q2: How can you make them more “stable” against Example: Base your activation and retention KPIs on visits of If you are focusing on your onboarding experience (eg. services not communities) focus base them on the user registration date. * Schmutz (noun, german/jiddish) means “dirt”
  27. 27. THE METRICS FAIRY SAYS… METRICS NEED TO HuRT#LEANCAMP | @andreasklinger
  28. 28. METRICS NEED TO HuRTIf the metrics you focus on do not hurt youeverytime you look at them and you are too embarassed to show them around. They are either a) not narrowed down enough or b) nothing you should focus on.
  29. 29. START WITH AARRREXAMPLE - LOOKK Nice to look at. useless to work with.ACQuISITION - user registered Eg.ACTIvATION - user voted (eg 90%) We changed activiation from “Registered users that voted”RETENTION - user visited Again toREFERRAL - … “Registered users that votedREvENuE - … for more than two designers” Everybody else is just voting for a friend. Now: AARRRR = THE PAIN THE PAIN = ACTIONABLE
  30. 30. “AARRR” IS THE SOuND OF PAIN ACQuISITION You need a way to reach the user. (Dave is wrong. A visit is just fog.s) ACTIvATION The user has seen the real core of your product and used it. (segment in user groups if multifaced) RETENTIONThe user did an action that leads to value. (if possible close to money or coreaction) REFERRAL Not only shared but invited users came. REvENuE Money.
  31. 31. TOO MANY METRICS ARE BAD FOR YOuR EYES FOCuS#LEANCAMP | @andreasklinger
  32. 32. TOO MANY METRICS ARE BAD FOR YOuR EYES On your dashboards only watch those metrics you are currently iterating with your product on. There are always numbers going up and Don’t go crazy because of that. Focus your product discovery around assumptions. Align your metrics to that.Example: At LOOKK we removed revenue from our dashboards while focusing on newsfeed and community features.
  33. 33. LESSONS LEARNED TOOLS#LEANCAMP | @andreasklinger
  34. 34. TOOLSTL;DR: All Suck.
  35. 35. TOOLSTL;DR: All Suck.…but are useful for certain parts:
  36. 36. TOOLSTL;DR: All Suck.…but are useful for certain parts:Kissmetrics, Mixpanel = People AnalysisYour own Database = drilldownBe consistent throughout all tools:- Naming of goals- Logging and Naming of user types (eg. user, Designer, Admin, visitor)
  37. 37. TOOLSTL;DR: All Suck.…but are useful for certain parts:Kissmetrics, Mixpanel = People AnalysisYour own Database = drilldownThere are three things you want to do:* Insights (how many…)* Accounting (compare birdeye weeks/months)* Comparing (eg. A/B or focused cohorts)
  38. 38. TOOLSgoogleAnalytics - Referrals - Optimize Online Marketing, - Bird eye view Website - Suck for funnel analysis (no ad-hoc eg) - Suck for consistancy (goals are easy to break) Don’t do funnel testing in gA - only track the last step Dashboards rock. Don’t go too crazy with customization Unfortunately single - it will break. player only (yet) READ: Cohorts in google Analytics by http://danhilltech.tumblr.com/
  39. 39. TOOLSKissMetrics - Insights on Conversions - Quick reports on features/pages - Suck (yet) at dashboards use it for funnel optimization. “View this people” feature is genius If you are a SaaS use it for everything ;) Don’t expect too much and also test mixpanel.
  40. 40. TOOLSLOOKKButler - SQL = quick, Tables = cheap - You can drilldown to your actionable KPIs - Dashboards for team members - Suck at bird-eye view, not ad- Questions: Who are the most popular designers if you only count people who vote for more than one designer? users?
  41. 41. TOOLSLOOKKButler Team dashboards are easy to do
  42. 42. BuT… ALWAYS REMEMBER IN PRE-PMF CuSTDEv > METRICS#LEANCAMP | @andreasklinger
  43. 43. THANKS. PLEASE SHARE YOuR LESSONS @ANDREASKLINgERPresentation slides: http://slideshare.net/andreasklinger
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×