Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Using data to inform
product decisions
Why we do need data to inform product development	

ProductTank Cape Town - 26 Marc...
Imagine this ...
... or this
Outline
Why
Driven vs Informed
What
5 things to be mindful of
Why do we need data?
Why do we need data?
What data do we need?
It’s about asking the right questions, you fool!!
Data & the product lifecycle
What do we want?
Assumption: “Sitting on a tractor all day isn’t the best use of
my time”
Assumption: My users want to spe...
What do we want?
Hypothesis: We believe this is true if the users of our
MVP spend 20% more time on the farm
Approaches: O...
What do we want?
What do we want?
How should it work?
How should it work?
Question: How can we encourage people to discover and
configure multiple cars?
Assumption: People will ...
How should it work?
Hypothesis: We believe that adding images will drive car
discovery. We know this is true if there’s a ...
“Blank slate”
“Blank slate”
One Single Metric: Percentage of users per variant who
configure another car
Design and sample size: Minimum ...
How is it working?
How is it working?
Question: Is our product / feature meeting the hypothesis?
Assumption: We believe that this feature wil...
How is it working?
Hypothesis: We know that our assumption is correct	

if we see a 20% increase (on the current benchmark...
How is it working?
Gathering the right data
What can quant data tell us?
What can quant data tell us?
Analytics
What can qual data tell us?
Qualitative data can help us:	

!
Understand the why behind quantitative data	

!
Get insight ...
What can qual data tell us?
What can qual data tell us?
What can qual data tell us?
Data driven
Data driven
A/B or multi-variate test continuously	

!
Focus on the “One Metric That Matters”	

!
Build hypothesis around ...
BUT...What data cannot tell
Is it a good product idea?	

!
Metrics do not always offer you the full
picture	

!
Data is on...
Data informed
Data informed
Data
Users
Intuition
Competition
Technology
Brand
Strategy
Business
Regulation
Time
Data informed
Data is one of the factors to consider	

!
Focus on the questions that you want
answered	

!
You cannot repl...
5 things to be mindful of
Focus on asking the right questions	

!
Data can’t replace intuition	

!
Be clear on hypothesis,...
SO ...
Embrace the data, don’t fear it!
Thanks!
marcabrahamlondon@gmail.com	

!
marcabraham.wordpress.com	

!
@MAA1	

!
!
!
Related links
http://svpg.com/assessing-product-opportunities/	

!
http://www.romanpichler.com/blog/goal-oriented-agile-pr...
Related links
!
https://marcabraham.wordpress.com/2015/03/05/what-is-guerrilla-testing/	

http://www.slideshare.net/LilyDa...
Upcoming SlideShare
Loading in …5
×

Using Data To Inform Product Decisions - Cape Town, 26 March '15

585 views

Published on

Aimed at product managers, this talk is about how to best use data to inform product decisions.

Published in: Data & Analytics
  • Be the first to comment

Using Data To Inform Product Decisions - Cape Town, 26 March '15

  1. 1. Using data to inform product decisions Why we do need data to inform product development ProductTank Cape Town - 26 March ‘15
  2. 2. Imagine this ...
  3. 3. ... or this
  4. 4. Outline Why Driven vs Informed What 5 things to be mindful of
  5. 5. Why do we need data?
  6. 6. Why do we need data?
  7. 7. What data do we need? It’s about asking the right questions, you fool!!
  8. 8. Data & the product lifecycle
  9. 9. What do we want? Assumption: “Sitting on a tractor all day isn’t the best use of my time” Assumption: My users want to spend less time on the tractor so that they can spend more time on other tasks Question: Is there a market need for driverless tractors?
  10. 10. What do we want? Hypothesis: We believe this is true if the users of our MVP spend 20% more time on the farm Approaches: One Single Metric, prototypes, MVP, observations, market research, diary studies
  11. 11. What do we want?
  12. 12. What do we want?
  13. 13. How should it work?
  14. 14. How should it work? Question: How can we encourage people to discover and configure multiple cars? Assumption: People will be encouraged to explore multiple cars if they see nice images of cars similar to the one they have just configured
  15. 15. How should it work? Hypothesis: We believe that adding images will drive car discovery. We know this is true if there’s a 30% increase in the average number of cars configured per person by end of May ’15 Approaches: A/B and MVT, behavioural plan & KPIs, prototypes and usability testing Approaches: A/B and MVT, behavioural plan & KPIs, prototypes and usability testing
  16. 16. “Blank slate”
  17. 17. “Blank slate” One Single Metric: Percentage of users per variant who configure another car Design and sample size: Minimum of 200 conversions per page to reach “statistical significance” BUT: I can’t learn everything through this experiment!
  18. 18. How is it working?
  19. 19. How is it working? Question: Is our product / feature meeting the hypothesis? Assumption: We believe that this feature will be used by 50% of our first time car buyers in the UK within the first month after release Question: What is our strongest market or user segment?
  20. 20. How is it working? Hypothesis: We know that our assumption is correct if we see a 20% increase (on the current benchmark) in the number of UK first time car buyers purchasing a car through our site Approaches: Usage tracking, user testing, product retrospectives and refine or reject hypothesis Identify opportunities for product improvement or reasons for discontinuation
  21. 21. How is it working?
  22. 22. Gathering the right data
  23. 23. What can quant data tell us?
  24. 24. What can quant data tell us?
  25. 25. Analytics
  26. 26. What can qual data tell us? Qualitative data can help us: ! Understand the why behind quantitative data ! Get insight into what people think and feel ! Learn about a product idea or prototype ! !
  27. 27. What can qual data tell us?
  28. 28. What can qual data tell us?
  29. 29. What can qual data tell us?
  30. 30. Data driven
  31. 31. Data driven A/B or multi-variate test continuously ! Focus on the “One Metric That Matters” ! Build hypothesis around key KPI ! Optimise your product based on data ! Are we making a noticeable difference?
  32. 32. BUT...What data cannot tell Is it a good product idea? ! Metrics do not always offer you the full picture ! Data is one of the factors that feed into a decision ! We typically do not own all product decisions
  33. 33. Data informed
  34. 34. Data informed Data Users Intuition Competition Technology Brand Strategy Business Regulation Time
  35. 35. Data informed Data is one of the factors to consider ! Focus on the questions that you want answered ! You cannot replace intuition or creative ideas with data ! Assess impact on relevant areas
  36. 36. 5 things to be mindful of Focus on asking the right questions ! Data can’t replace intuition ! Be clear on hypothesis, sample size and timings ! Build and launch with data in mind ! Listen to the data and act accordingly!
  37. 37. SO ... Embrace the data, don’t fear it!
  38. 38. Thanks! marcabrahamlondon@gmail.com ! marcabraham.wordpress.com ! @MAA1 ! ! !
  39. 39. Related links http://svpg.com/assessing-product-opportunities/ ! http://www.romanpichler.com/blog/goal-oriented-agile-product-roadmap/ http://vimeo.com/14999991 http://www.realityisagame.com/archives/390/wooga-follows-zynga-in-metrics-driven-game-design/ http://marcabraham.wordpress.com/2013/05/03/book-review-lean-analytics/ http://www.kaushik.net/avinash/web-analytics-101-definitions-goals-metrics-kpis-dimensions- targets/ http://marcabraham.wordpress.com/2013/09/09/some-considerations-regarding-data-driven- design/ http://insideintercom.io/the-problem-with-data-driven-decisions/ http://www.webdesignerdepot.com/2013/05/the-perils-of-ab-testing/ http://andrewchen.co/2008/09/08/how-to-measure-if-users-love-your-product-using-cohorts-and- revisit-rates/ http://codeascraft.com/2012/06/21/building-websites-with-science/
  40. 40. Related links ! https://marcabraham.wordpress.com/2015/03/05/what-is-guerrilla-testing/ http://www.slideshare.net/LilyDart/guerrilla-testing-for-content https://marcabraham.wordpress.com/2015/02/14/learning-more-about-running-ab-tests/ https://marcabraham.wordpress.com/2015/02/04/book-review-thinking-with-data/ https://marcabraham.wordpress.com/2015/01/26/book-review-designing-for-behavior-change/ http://www.simplypsychology.org/qualitative-quantitative.html http://data.heapanalytics.com/dont-stop-your-ab-tests-part-way-through/ https://marcabraham.wordpress.com/2014/12/22/book-review-web-metrics/ ! ! !

×