The document discusses lean analytics and lean principles for software development. It describes the main stages of product development - empathy, stickiness, virality, revenue, and scale. For each stage, it recommends key metrics to measure, such as daily/weekly active users for stickiness, viral coefficient for virality, and quarterly recurring revenue for revenue. It emphasizes focusing on one metric that matters the most for each stage, measuring assumptions in the right order, and using both qualitative and quantitative metrics for continuous learning and improvement.

1. LEAN ANALYTICS
techtalk @ ferret
based on “Lean Analytics” by Croll&Yoskovitz and wikipedia information

2. LEAN SOFTWARE DEVELOPMENT
The term lean software development originated
in a book “Lean Software Development: An Agile
Toolkit”, written by Mary Poppendieck and Tom
Poppendieck.

3. LEAN PRINCIPLES

4. •
Focus on the customer and eliminate waste
Everything not adding value to the customer is considered
to be waste. If some activity could be bypassed or the
result could be achieved without it, it is waste.
•
Amplify learning
The best approach for improving a software development
environment is to amplify learning. Instead of adding more
documentation or detailed planning, different ideas could
be tried by writing code and building. A data driven cycle
of hypothesis-validation-implementation should be used to
drive innovation and continuously improve the end-to-end
process.

5. •
Decide as late as possible
Delay decisions as much as possible until they can be
made based on facts and not on uncertain assumptions
and predictions.
•
Deliver as fast as possible
The sooner the end product is delivered without
major defects, the sooner feedback can be received,
and incorporated into the next iteration. The shorter
the iterations, the better the learning and
communication within the team.

6. •
Empower the team
“Find good people and let them do their own job”.
People do need something more than just the
list of tasks and the assurance that they will not
be disturbed during the completion of the
tasks. People need motivation. The developers
should be given access to the customer; the
team leader should provide support and help
in difficult situations.

7. •
Build integrity in
Understanding the problem domain and solving it at
the same time, not sequentially. The information flow
should be constant in both directions – from customer
to developers and back.
•
See the whole
“Think big, act small, fail fast; learn rapidly”. Larger
software = more part developed by different teams,
but lean thinking has to be understood well by all
members of a project, before implementing in a
concrete, real-life situation.

8. “If you can’t measure something, you can’t manage it.”
– Peter Drucker, management consultant

9. LINE IN THE SAND
Measurable target that everyone (incl. executives)
agrees to.

10. GOOD METRICS
• Qualitative
vs. quantitative
quantitative data answers “what” and “how much”, qualitative
data answers “why”. Quantitative data has no emotions.
• Vanity
vs. actionable
Use metrics you can act on. Vanity metrics might make you
feel good, but they don’t let you act. For instance, “total
signup” vs. “percent of active users”.

11. • Exploratory
vs. reportable
Exploratory metrics are speculative, while reporting metrics keep
you nearby of normal, day-to-day operations.
• Leading
vs. lagging
Leading metrics give you predictive understanding of the future,
while lagging metrics explain the past. Leading metrics are better
because you have time to act on them.
• Correlated
vs. causal
If 2 metrics change together, they’re correlated, but if one causes
another to change, they’re causal. Try to find a causal relationship
between smth. you want and smth. you can control.

12. “Acquisition, activation, retention, revenue, and referral
- AARRR.”
–Pirate Metrics by Dave McClure that every startup needs to watch

13. RIGHT METRIC FOR RIGHT NOW
•
Acquisition
How do users aware of you? Metrics: traffic, mentions,
cost per click, search results, cost of acquisition, open rate.
•
Activation
Do drive-by visitors subscribe, use etc.? Metrics: signups,
completed on boarding process, used service at least
once, subscriptions

14. •
Retention
Does a one-time user become engaged? Metrics:
engagement, time since last visit, daily/monthly active use,
churns
•
Revenue
Do you make money from user activity? Metrics: customer
lifetime value, conversion rate, shopping cart size, clickthrough revenue
•
Referral
Do users promote you product? Metrics: invites sent, viral
coefficient, viral cycle time

15. ONE METRIC THAT MATTERS
The OMTM is the one number you’re completely focused on
above everything else for your current stage.

16. •
It answers the most important question you have
you need to identify the riskiest areas of your business, and that’s
where the most important question is.
•
It forces you to draw a line in the sand and have clear goals
after you’ve identified the key problem, you need to set goals.
•
It focuses the entire company
Use OMTM as a way of focusing you entire company. Display it
throughout web dashboards, on TV screens, or regular emails.
•
It inspires a culture of experimentation
It’s critical to move through the build-measure-learn cycle as quickly
and as frequently as possible. To succeed on that, you need to
actively encourage experimentation.

17. the One Metric That Matters changes over time
When you are focused on retention, you may be looking on churn, and
experimenting with pricing, features, improving customer support etc.

18. MAIN STAGES
You can’t just start measuring at once. You have to measure
your assumptions in the right order.

19. •
Empathy
Go inside target market and sure you’re solving
a problem people care about in a way
someone will pay for.
•
Stickiness
It comes from a good product. You need to
find out if you can build an acceptable solution
to the problem you’ve discovered.

20. •
Virality
Once you’ve got a product that’s sticky, you need care
about acquisition.
•
Revenue
You’re giving away free trials, free drinks, or free
copies. Now you’re focused on maximising and
optimising revenue.
•
Scale
With revenues coming in, it’s time to move from
growing your business to growing you market.

21. EMPATHY

22. •
Find a problem to fix:
- The problem painful enough
- Enough people care
- They are already trying to solve it
•
Talk with people and rate interviews
•
Mostly qualitative metrics here. Be honest with
yourself.

23. STICKINESS

24. •
Daily, weekly, monthly active users
•
Time to become inactive
•
Number of reactivated inactive after email
•
Time to spend with feature

25. 7 QUESTIONS TO ASK BEFORE
BUILDING A FEATURE
•
Why will it make things better?
You can’t build a feature without a reason. Ask
yourself “Why will it make it better?” and write out
a hypothesis.

26. •
Can you measure the effect of the feature?
Feature experiments require that you measure
the impact of the feature. That impact has to
be quantifiable.
•
How long will the feature take to build?
Time is resource you never get back. If
something is going to take too much to build,
break it into small parts or test the risk with
the prototype first.

27. •
Will the future over complicate things?
Complexity kills products. When discussing a
feature with your team, pay attention to how
it’s being described. “And” is enemy of success.
•
How much risk is there in this feature?
Building a new feature always comes with
some amount of risks - technical risk, user risk,
risk of influence to further development etc.

28. •
How innovative is the new feature?
Not everything is innovative, but consider
innovation when prioritising feature
development. Generally, the easiest thing to do
rarely have a big impact.
•
What do users say they want?
Users are important as well as their feedback.
But relying on what they say is risky. Be careful
about over prioritising based on user input alone.

29. VIRALITY

30. •
Invitation rate - the number of invites sent
divided by the number of users you have
•
Acceptance rate - the number of signups or
enrolments divided by the number of invites
•
Viral coefficient (OMTM) - the number of
new customers that each existing customer is
able to successfully convert
Viral = invitation rate x acceptance rate

31. REVENUE

32. •
QRR(x) - the quarterly recurring revenue for
quarter x
•
QExpSM(x) - sales and marking expense for the
quarter x
•
Ratio of inputs to outputs (OMTM)
q = [QRR(x)-QRR(x-1)] / QExpSM(x-1)
You have problems if q < 0.75

33. SCALE

34. •
On this stage you already know your product and market.
Your metrics now should be focused on the health of
your ecosystem and your ability to enter new markets.
•
Customer acquisition payback (OMTM) - the
customer acquisition cost divided by the customer
lifetime value
•
Check metrics across channels, regions, and marketing
campaigns
•
Try to understand if you’re focused on efficiency(try to
reduce cost) or differentiation (try to increase margins).

35. CONCLUSION

36. •
Make Sure Goals are Clearly Understood
To prove the value of analytic-focused company, any
project needs to have clear goals. Everyone involved in the
project needs to be aligned around the goals.
•
Make Things Simple to Digest
A good metric is the one that’s easy to understand at
glance. Metric can be extremely valuable, but used
incorrectly they will lead down the wrong path.
•
Ensure Transparency
If you are going to use data to make decisions, it’s
important that you share the data and methodologies.

37. •
Don’t Eliminate your Gut
Lean Analytics isn’t about eliminating your gut,
it’s about proving your gut right or wrong.
•
Ask Good Questions
You don’t need to guess, you need to know
where to focus. You don’t know all answers, but
you should know the right questions to ask.

38. FUTHER READINGS
•
Lean Software Development: An Agile Toolkit by
Mary & Tom Poppendieck
•
Lean Analytics by A. Croll, B. Yoskovitz
•
The Lean Startup by Eric Ries
•
Running Lean by Ash Maurya