Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

7 Practical Tips on Lean Analytics

344 views

Published on

Experiences from defining lean analytics and for running AB tests in a web startup.

1. Lean Analytics requires a Lean project
2. Specify success metric for every feature
3. You need lots of users for an A/B test
4. Make big changes
5. Measure your tests correctly
6. Push through radical changes with A/B
7. Use the right tools

Published in: Software
  • Be the first to comment

  • Be the first to like this

7 Practical Tips on Lean Analytics

  1. 1. 7 Practical Tips on Lean Analytics Janne Aukia — 17 Feb 2015
  2. 2. 1. Lean Analytics requires a Lean project Illustration from commons.wikimedia.org/wiki/File:Kepler-solar-system-1.png
  3. 3. Challenging for a consultancy? Lean Analytics works well with Lean Startup, Lean UX and
 Agile projects Question designs, question authorities, ask difficult questions, be prepared to kill features. Requires freedom to experiment
  4. 4. 2. Write down a success metric for every feature
  5. 5. Was this feature used more than Google+ login? If feature isn’t successful, 
 just kill it. Measure only actionable metrics. Is the feature for acquisition, retention or revenue?
  6. 6. 3. You need lots of users for an A/B test
  7. 7. Typically at least 10k events, such as visits or people using a feature. Doesn’t work for small startups.
  8. 8. 4. Make big changes Illustration from http://quod.lib.umich.edu/m/moa/agb8710.0001.001/26
  9. 9. If you cannot see the changes from 2 meters away, most likely your AB test won’t either. Google A/B tested 41 shades of blue. Don’t do that — leads to local optimum. Illustration from http://www.doxsey.net/blog/too-practically-minded-to-be-of-any-theological-good/
  10. 10. 5. Measure your tests correctly
  11. 11. Never terminate early
 a running test Use a split test calculator, 
 for example: http://getdatadriven.com/ab-significance-test
  12. 12. 6. Push through radical changes with A/B
  13. 13. 7. Use the right tools
  14. 14. Google Analytics can be used for tracking events but it is painful. Mixpanel more suited for the job. Consider Keen.io, Heap as well.
  15. 15. 1.Lean Analytics requires a Lean project 2.Specify success metric for every feature 3.You need lots of users for an A/B test 4.Make big changes 5.Measure your tests correctly 6.Push through radical changes with A/B 7.Use the right tools Summary

×