Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Lessons about experiments

2,090 views

Published on

Here is how we approach experiments @ Vinted as well as what we have learnt during few years of practising it.

Published in: Business
  • Login to see the comments

Lessons about experiments

  1. 1. SPEAKER What we have learnt @Vinted Experiments Tautvydas Gylys
  2. 2. Experiments - scientific approach Observe > Hypothesis > Test > Prove / Reject photo:
  3. 3. 2 types of experimentation Conversion optimisation
 Hypothesis driven development To improve smth To create smth new Will be talking about today
  4. 4. How do we experiment @Vinted? case study: Conversion to lister optimisation
  5. 5. case study: Conversion to lister optimisation
  6. 6. case study: Conversion to lister optimisation
  7. 7. 2. Ideas and assumptions Data analysis
  8. 8. 2. Ideas and assumptions Interviews Data analysis
  9. 9. 2. Ideas and assumptions Interviews Data analysis Brainstorm
  10. 10. case study: Conversion to lister optimisation
  11. 11. case study: Conversion to lister optimisation
  12. 12. 4. Experiment design
  13. 13. 4. Experiment design
  14. 14. case study: Conversion to lister optimisation
  15. 15. 7. Analysis and documentation Dashboard v1
  16. 16. Dashboard v2 7. Analysis and documentation
  17. 17. results: Listers flow optimisation ~30 ideas 7 experiments 2 implemented Learning!
  18. 18. Required registration example
  19. 19. Conversion to lister: - Enter product > Register > Listing form
  20. 20. Conversion to lister: Enter product > Listing form > Register on upload
  21. 21. Conversion to lister: Register > Enter product > Listing form
  22. 22. Conversion to lister: Required registration V2
  23. 23. Required registration V2 Conversion to lister: Take a way: in apps members are willing to invest time upfront to register. That way they feel invested and explore the product with more attention.
  24. 24. Lessons learned
  25. 25. Idea: change to 
 “Buy now” Goal: to improve conversion to buyer Is it a good idea?
  26. 26. Idea: change to 
 “Buy now” Goal: improve conversion to buyer Goal to increase conversion by 3% Goal to increase conversion by 2x Yes! No! Is it a good idea?
  27. 27. Idea: change to 
 “Buy now” Goal: improve conversion to buyer Goal to increase conversion by 3% Goal to increase conversion by 2x Yes! No! Is it a good idea? Always have measurable goals before designing an experiment.
  28. 28. How many variables? Control: Not required Test: Required
  29. 29. How many variables? Control: Not required Test: Required Picture Copy CTA Screen size Screen number
  30. 30. Control Modal screen / Not required Test A Modal screen / Required Test B Full screen / Not required Test C Full screen / Required
  31. 31. Control Modal screen / Not required Test A Modal screen / Required Test B Full screen / Not required Test C Full screen / Required Be aware of how many variables you are testing.
  32. 32. Experiment culture changes • Before: Dedicated experiment team (I have an idea, let’s test it!) • Now: Each team runs experiments (I have a problem which potential solution I test)
  33. 33. Experiment culture changes • Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test) • Before: Test everything • Now: Test only big things, trust gut feeling for small
  34. 34. Experiment culture changes • Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test) • Before: Test everything • Now: Test only big things, trust gut feeling for small • Before: Data only + Gut feeling • Now: Data + Interviews + Gut feeling
  35. 35. Experiment culture changes • Before: Dedicated experiment team (I have an idea, lets test it!) • Now: Each team runs experiments (I have a problem which potential solution I test) • Before: Test everything • Now: Test only big things, trust gut feeling for small • Before: Data only + Gut feeling • Now: Data + Interviews + Gut feeling • Before: Dig deep to learn the impact • Now: Fail fast
  36. 36. Thanks!

×