Successfully reported this slideshow.
Your SlideShare is downloading. ×

Using Data To Inform Product Decisions - Cape Town, 26 March '15

Ad

Using data to inform
product decisions
Why we do need data to inform product development	

ProductTank Cape Town - 26 Marc...

Ad

Imagine this ...

Ad

... or this

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Ad

Check these out next

1 of 40 Ad
1 of 40 Ad
Advertisement

More Related Content

Advertisement
Advertisement

Using Data To Inform Product Decisions - Cape Town, 26 March '15

  1. 1. Using data to inform product decisions Why we do need data to inform product development ProductTank Cape Town - 26 March ‘15
  2. 2. Imagine this ...
  3. 3. ... or this
  4. 4. Outline Why Driven vs Informed What 5 things to be mindful of
  5. 5. Why do we need data?
  6. 6. Why do we need data?
  7. 7. What data do we need? It’s about asking the right questions, you fool!!
  8. 8. Data & the product lifecycle
  9. 9. What do we want? Assumption: “Sitting on a tractor all day isn’t the best use of my time” Assumption: My users want to spend less time on the tractor so that they can spend more time on other tasks Question: Is there a market need for driverless tractors?
  10. 10. What do we want? Hypothesis: We believe this is true if the users of our MVP spend 20% more time on the farm Approaches: One Single Metric, prototypes, MVP, observations, market research, diary studies
  11. 11. What do we want?
  12. 12. What do we want?
  13. 13. How should it work?
  14. 14. How should it work? Question: How can we encourage people to discover and configure multiple cars? Assumption: People will be encouraged to explore multiple cars if they see nice images of cars similar to the one they have just configured
  15. 15. How should it work? Hypothesis: We believe that adding images will drive car discovery. We know this is true if there’s a 30% increase in the average number of cars configured per person by end of May ’15 Approaches: A/B and MVT, behavioural plan & KPIs, prototypes and usability testing Approaches: A/B and MVT, behavioural plan & KPIs, prototypes and usability testing
  16. 16. “Blank slate”
  17. 17. “Blank slate” One Single Metric: Percentage of users per variant who configure another car Design and sample size: Minimum of 200 conversions per page to reach “statistical significance” BUT: I can’t learn everything through this experiment!
  18. 18. How is it working?
  19. 19. How is it working? Question: Is our product / feature meeting the hypothesis? Assumption: We believe that this feature will be used by 50% of our first time car buyers in the UK within the first month after release Question: What is our strongest market or user segment?
  20. 20. How is it working? Hypothesis: We know that our assumption is correct if we see a 20% increase (on the current benchmark) in the number of UK first time car buyers purchasing a car through our site Approaches: Usage tracking, user testing, product retrospectives and refine or reject hypothesis Identify opportunities for product improvement or reasons for discontinuation
  21. 21. How is it working?
  22. 22. Gathering the right data
  23. 23. What can quant data tell us?
  24. 24. What can quant data tell us?
  25. 25. Analytics
  26. 26. What can qual data tell us? Qualitative data can help us: ! Understand the why behind quantitative data ! Get insight into what people think and feel ! Learn about a product idea or prototype ! !
  27. 27. What can qual data tell us?
  28. 28. What can qual data tell us?
  29. 29. What can qual data tell us?
  30. 30. Data driven
  31. 31. Data driven A/B or multi-variate test continuously ! Focus on the “One Metric That Matters” ! Build hypothesis around key KPI ! Optimise your product based on data ! Are we making a noticeable difference?
  32. 32. BUT...What data cannot tell Is it a good product idea? ! Metrics do not always offer you the full picture ! Data is one of the factors that feed into a decision ! We typically do not own all product decisions
  33. 33. Data informed
  34. 34. Data informed Data Users Intuition Competition Technology Brand Strategy Business Regulation Time
  35. 35. Data informed Data is one of the factors to consider ! Focus on the questions that you want answered ! You cannot replace intuition or creative ideas with data ! Assess impact on relevant areas
  36. 36. 5 things to be mindful of Focus on asking the right questions ! Data can’t replace intuition ! Be clear on hypothesis, sample size and timings ! Build and launch with data in mind ! Listen to the data and act accordingly!
  37. 37. SO ... Embrace the data, don’t fear it!
  38. 38. Thanks! marcabrahamlondon@gmail.com ! marcabraham.wordpress.com ! @MAA1 ! ! !
  39. 39. Related links http://svpg.com/assessing-product-opportunities/ ! http://www.romanpichler.com/blog/goal-oriented-agile-product-roadmap/ http://vimeo.com/14999991 http://www.realityisagame.com/archives/390/wooga-follows-zynga-in-metrics-driven-game-design/ http://marcabraham.wordpress.com/2013/05/03/book-review-lean-analytics/ http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions- targets/ http://marcabraham.wordpress.com/2013/09/09/some-considerations-regarding-data-driven- design/ http://insideintercom.io/the-problem-with-data-driven-decisions/ http://www.webdesignerdepot.com/2013/05/the-perils-of-ab-testing/ http://andrewchen.co/2008/09/08/how-to-measure-if-users-love-your-product-using-cohorts-and- revisit-rates/ http://codeascraft.com/2012/06/21/building-websites-with-science/
  40. 40. Related links ! https://marcabraham.wordpress.com/2015/03/05/what-is-guerrilla-testing/ http://www.slideshare.net/LilyDart/guerrilla-testing-for-content https://marcabraham.wordpress.com/2015/02/14/learning-more-about-running-ab-tests/ https://marcabraham.wordpress.com/2015/02/04/book-review-thinking-with-data/ https://marcabraham.wordpress.com/2015/01/26/book-review-designing-for-behavior-change/ http://www.simplypsychology.org/qualitative-quantitative.html http://data.heapanalytics.com/dont-stop-your-ab-tests-part-way-through/ https://marcabraham.wordpress.com/2014/12/22/book-review-web-metrics/ ! ! !

×