Developing KPIs for startups

What makes startups different to mature businesses, and what the implications
       are for the role of KPI and the approach to their development
KPIs have a different role in mature businesses versus
  startups. In a mature business, different teams should
  be responsible for different KPIs…
Illustrative

                   Customer Acquisition         Customer Retention               Customer Monetisation
      Team
                         team                         team                              team


                   • Optimise spend on each    • Understand drivers of           • Optimise promotions,
                     marketing channel           churn (price / competitor         customer email /
                   • Optimise spend across       promotion / service               activation, loyalty
 Objective and
                     marketing channels          levels / content) and act         schemes
  approach                                       on them                         • Drive down costs,
                                                                                   increase margin




                 • Number of customers        • Rate of repeat purchase /    •   Customer lifetime value
                   acquired in time period      repeat visits                •   ARPU
                 • Cost per acquisition                                      •   Basket size
       KPIs
                                                                             •   Revenue per sale
                                                                             •   Profit per sale
…whereas in a startup, the KPIs should focus on driving
     product-market fit, so that the startup gets into a
     “virtuous circle” as quickly as possible

                                 +                  Product meets customer need                +



                          ↑ # customers
                                                                                       Customers ♥ product
                         ↑ customer data
                                                                                             Churn ↓
                        ↑ customer insight
                                                                                           Retention ↑
                      ↑ product development

The larger the userbase, the
  more customer data the
  startup collects, and the                         Customers evangelise product
   better able they are to                                Cost of customer                                 If product-market fit is
 develop customer insight
    and use that to drive
                                 +                          acquisition↓                        +           not achieved, it does
                                                                                                            not matter how good
   product development.                                  # new customers ↑                                       the individual
 Customer data becomes a                                                                                     departmental teams
          key asset                                                                                        are: the startup will not
                                                                                                              grow into a viable
                                                                                                              commercial entity


                          KPIs are needed to drive product development decisions i.e. measure progress
                                              towards achieving product-market fit.
                         This means that startup KPIs focus on user behaviour more than traditional KPIs
Designing and implementing KPIs at a startup is a
two stage process
             1. Identify the success factors                                           2. Develop corresponding KPI

Identify the combination of success factors which drive                   For each success factor, identify the set of metrics that
product-market fit and thus build the virtuous circle:                    capture to what extent that is happening:

                                                                                 Success factor                        KPI
Example: Social network                                                             ‘Stickiness’              • # of logins per month
                                                                                                              • Actions per month
• Stickiness – what proportion of users continue to use the                                                   • Minutes logged in per
  service once being introduced to it                                                                           month
• Viral coefficient – how many ‘friends’ does each customer
  refer the product to                                                    Use the distribution of each metric to identify which are
• Profit per user – how much revenue is made per user,                    both robust (reliable) and sensitive (move with changes):
  relative to the cost per user                                                                    +
                                                                                                   +
                                                                                                   +++
                                                                                                      +
                                                                                                     ++
                                                                                                          + + + + ++
Example: Retailer
                                                                          Settle on ideally one KPI per success factor, or if necessary a
• Customer lifetime value – how much revenue a customer                   handful.
  generates over their lifetime
• Acquisition cost – how much it costs to acquire a new                   Establish the baseline and measure improvements from
  customer                                                                there
                                         Identifying the key success
                                        factors for Shopcade is not-
                                      trivial given its unique business
                                                    model
Good KPIs meet a number of requirements, most of
which are relatively straightforward to deliver

                                                                                        The set of KPIs should be...
                 Each KPI should be...                                              The set of KPIs shouldbe...
                                                                                      The set of KPIs should
                                                                                                             be...


•   Relevant: measure something that matters to the                   •   Comprehensive: if something is wrong with the
    business success                                                      business, it needs to be ‘picked up’ by at least one of the
     –    Each KPI should be associated with one of the                   KPIs
          identified ‘success factors’                                     –    There needs to be a KPI for each key success factor
                                                                                identified
•   Responsive: when things ‘go wrong’, the KPI value
    should change in a noticeable way, fast. Similarly, when          •   Actionable: it needs to be possible to make product
    things ‘go right’, the KPI value should quickly improve:              development decisions based on the combination of
     –    This makes cumulative metrics e.g. total user numbers           KPIs. This is not easy and is discussed on the following
          dangerous ‘vanity metrics’                                      slide…
     –    This makes time-based metrics (number of actions per
          month, number of signups per month) attractive


•   Easy to understand: minimise ambiguity
     –    It should be clear what a ‘good’ result is versus a ‘bad’
          result
     –    It should be clear how it is calculated, with no doubt
          about the data collection methodology or accuracy of
          the measure
To ensure KPIs are actionable, they need to be part of a
broader analytics effort at the heart of the formal
product development process
KPIs are at the centre of a broader analytics effort          KPIs provide an important ‘rearview’ check on the
                 that also includes:                                   product development process...


                 • Talking to customers is necessary to                          • Cohort analysis and/or split testing to
                   understand why their behaviours are           3. Formal         robustly measure what impact a given
                   changing, which needs are and are             analysis of       product development is having on the
1. Qualitative     not being met, and to get new ideas           impact of         KPI
research with      for product development                        product
                                                                                 • Formal review of these results as part
  customers                                                    developments
                 • This can take the form of surveys,                              of the agile development process
                                                                on the KPIs
                   customer interviews (telephone or in
                   person), focus groups, watching
                   customers use the service

                                                               ... and the broader analytics effort should drive
                  • Deep quantitative analysis helps to              product development going forwards
                    understand how customers are
                    actually using the product
 2. More in-                                                                     • The product team should use findings
    depth         • This can be used to identify e.g. where
                    they are getting stuck, or whether           4. Ongoing        from the qualitative research (box 1)
 quantitative                                                     review of        and the in-depth quantitative analysis
   analysis         they are using the product in a
                    different way than is intended                customer         (box 2) to propose, specify and
                                                               intelligence to     prioritise new product features as part
                                                              inform product       of the agile planning process
                                                                development

A KPI framework for startups

  • 1.
    Developing KPIs forstartups What makes startups different to mature businesses, and what the implications are for the role of KPI and the approach to their development
  • 2.
    KPIs have adifferent role in mature businesses versus startups. In a mature business, different teams should be responsible for different KPIs… Illustrative Customer Acquisition Customer Retention Customer Monetisation Team team team team • Optimise spend on each • Understand drivers of • Optimise promotions, marketing channel churn (price / competitor customer email / • Optimise spend across promotion / service activation, loyalty Objective and marketing channels levels / content) and act schemes approach on them • Drive down costs, increase margin • Number of customers • Rate of repeat purchase / • Customer lifetime value acquired in time period repeat visits • ARPU • Cost per acquisition • Basket size KPIs • Revenue per sale • Profit per sale
  • 3.
    …whereas in astartup, the KPIs should focus on driving product-market fit, so that the startup gets into a “virtuous circle” as quickly as possible + Product meets customer need + ↑ # customers Customers ♥ product ↑ customer data Churn ↓ ↑ customer insight Retention ↑ ↑ product development The larger the userbase, the more customer data the startup collects, and the Customers evangelise product better able they are to Cost of customer If product-market fit is develop customer insight and use that to drive + acquisition↓ + not achieved, it does not matter how good product development. # new customers ↑ the individual Customer data becomes a departmental teams key asset are: the startup will not grow into a viable commercial entity KPIs are needed to drive product development decisions i.e. measure progress towards achieving product-market fit. This means that startup KPIs focus on user behaviour more than traditional KPIs
  • 4.
    Designing and implementingKPIs at a startup is a two stage process 1. Identify the success factors 2. Develop corresponding KPI Identify the combination of success factors which drive For each success factor, identify the set of metrics that product-market fit and thus build the virtuous circle: capture to what extent that is happening: Success factor KPI Example: Social network ‘Stickiness’ • # of logins per month • Actions per month • Stickiness – what proportion of users continue to use the • Minutes logged in per service once being introduced to it month • Viral coefficient – how many ‘friends’ does each customer refer the product to Use the distribution of each metric to identify which are • Profit per user – how much revenue is made per user, both robust (reliable) and sensitive (move with changes): relative to the cost per user + + +++ + ++ + + + + ++ Example: Retailer Settle on ideally one KPI per success factor, or if necessary a • Customer lifetime value – how much revenue a customer handful. generates over their lifetime • Acquisition cost – how much it costs to acquire a new Establish the baseline and measure improvements from customer there Identifying the key success factors for Shopcade is not- trivial given its unique business model
  • 5.
    Good KPIs meeta number of requirements, most of which are relatively straightforward to deliver The set of KPIs should be... Each KPI should be... The set of KPIs shouldbe... The set of KPIs should be... • Relevant: measure something that matters to the • Comprehensive: if something is wrong with the business success business, it needs to be ‘picked up’ by at least one of the – Each KPI should be associated with one of the KPIs identified ‘success factors’ – There needs to be a KPI for each key success factor identified • Responsive: when things ‘go wrong’, the KPI value should change in a noticeable way, fast. Similarly, when • Actionable: it needs to be possible to make product things ‘go right’, the KPI value should quickly improve: development decisions based on the combination of – This makes cumulative metrics e.g. total user numbers KPIs. This is not easy and is discussed on the following dangerous ‘vanity metrics’ slide… – This makes time-based metrics (number of actions per month, number of signups per month) attractive • Easy to understand: minimise ambiguity – It should be clear what a ‘good’ result is versus a ‘bad’ result – It should be clear how it is calculated, with no doubt about the data collection methodology or accuracy of the measure
  • 6.
    To ensure KPIsare actionable, they need to be part of a broader analytics effort at the heart of the formal product development process KPIs are at the centre of a broader analytics effort KPIs provide an important ‘rearview’ check on the that also includes: product development process... • Talking to customers is necessary to • Cohort analysis and/or split testing to understand why their behaviours are 3. Formal robustly measure what impact a given changing, which needs are and are analysis of product development is having on the 1. Qualitative not being met, and to get new ideas impact of KPI research with for product development product • Formal review of these results as part customers developments • This can take the form of surveys, of the agile development process on the KPIs customer interviews (telephone or in person), focus groups, watching customers use the service ... and the broader analytics effort should drive • Deep quantitative analysis helps to product development going forwards understand how customers are actually using the product 2. More in- • The product team should use findings depth • This can be used to identify e.g. where they are getting stuck, or whether 4. Ongoing from the qualitative research (box 1) quantitative review of and the in-depth quantitative analysis analysis they are using the product in a different way than is intended customer (box 2) to propose, specify and intelligence to prioritise new product features as part inform product of the agile planning process development